2023-08-06 09:53:09,874 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-08-06 09:53:10,115 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f 2023-08-06 09:53:10,129 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-06 09:53:10,129 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2, deleteOnExit=true 2023-08-06 09:53:10,130 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-06 09:53:10,130 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/test.cache.data in system properties and HBase conf 2023-08-06 09:53:10,131 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/hadoop.tmp.dir in system properties and HBase conf 2023-08-06 09:53:10,131 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/hadoop.log.dir in system properties and HBase conf 2023-08-06 09:53:10,132 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-06 09:53:10,132 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-06 09:53:10,132 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-06 09:53:10,253 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-08-06 09:53:10,721 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-06 09:53:10,727 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-06 09:53:10,727 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-06 09:53:10,728 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-06 09:53:10,728 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-06 09:53:10,728 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-06 09:53:10,729 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-06 09:53:10,729 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-06 09:53:10,729 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-06 09:53:10,730 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-06 09:53:10,730 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/nfs.dump.dir in system properties and HBase conf 2023-08-06 09:53:10,731 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/java.io.tmpdir in system properties and HBase conf 2023-08-06 09:53:10,731 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-06 09:53:10,731 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-06 09:53:10,732 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-06 09:53:11,376 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-06 09:53:11,381 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-06 09:53:11,720 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-08-06 09:53:11,942 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-08-06 09:53:11,969 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-06 09:53:12,014 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-08-06 09:53:12,050 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/java.io.tmpdir/Jetty_localhost_42057_hdfs____.ylw1xe/webapp 2023-08-06 09:53:12,222 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42057 2023-08-06 09:53:12,241 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-06 09:53:12,241 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-06 09:53:12,807 WARN [Listener at localhost/43345] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-06 09:53:12,892 WARN [Listener at localhost/43345] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-06 09:53:12,916 WARN [Listener at localhost/43345] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-06 09:53:12,923 INFO [Listener at localhost/43345] log.Slf4jLog(67): jetty-6.1.26 2023-08-06 09:53:12,927 INFO [Listener at localhost/43345] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/java.io.tmpdir/Jetty_localhost_39663_datanode____.d0mks/webapp 2023-08-06 09:53:13,027 INFO [Listener at localhost/43345] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39663 2023-08-06 09:53:13,414 WARN [Listener at localhost/38125] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-06 09:53:13,465 WARN [Listener at localhost/38125] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-06 09:53:13,471 WARN [Listener at localhost/38125] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-06 09:53:13,482 INFO [Listener at localhost/38125] log.Slf4jLog(67): jetty-6.1.26 2023-08-06 09:53:13,487 INFO [Listener at localhost/38125] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/java.io.tmpdir/Jetty_localhost_32837_datanode____.rs5x30/webapp 2023-08-06 09:53:13,620 INFO [Listener at localhost/38125] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32837 2023-08-06 09:53:13,642 WARN [Listener at localhost/38769] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-06 09:53:13,707 WARN [Listener at localhost/38769] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-06 09:53:13,710 WARN [Listener at localhost/38769] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-06 09:53:13,712 INFO [Listener at localhost/38769] log.Slf4jLog(67): jetty-6.1.26 2023-08-06 09:53:13,724 INFO [Listener at localhost/38769] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/java.io.tmpdir/Jetty_localhost_36593_datanode____.p1oril/webapp 2023-08-06 09:53:13,855 INFO [Listener at localhost/38769] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36593 2023-08-06 09:53:13,877 WARN [Listener at localhost/36789] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-06 09:53:14,105 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xefbdcbaab02fe3be: Processing first storage report for DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4 from datanode 66ee7809-9841-425b-bf64-0316ea0c97c6 2023-08-06 09:53:14,107 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xefbdcbaab02fe3be: from storage DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4 node DatanodeRegistration(127.0.0.1:44229, datanodeUuid=66ee7809-9841-425b-bf64-0316ea0c97c6, infoPort=39943, infoSecurePort=0, ipcPort=38769, storageInfo=lv=-57;cid=testClusterID;nsid=1360210479;c=1691315591457), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-08-06 09:53:14,107 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4f7e8d476b04a77b: Processing first storage report for DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d from datanode d309b271-e4de-4313-87b3-7ace32969fdc 2023-08-06 09:53:14,107 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4f7e8d476b04a77b: from storage DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d node DatanodeRegistration(127.0.0.1:43019, datanodeUuid=d309b271-e4de-4313-87b3-7ace32969fdc, infoPort=37633, infoSecurePort=0, ipcPort=38125, storageInfo=lv=-57;cid=testClusterID;nsid=1360210479;c=1691315591457), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-06 09:53:14,107 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x797192cc284db483: Processing first storage report for DS-47ea95e7-b60e-4e13-a055-e60cc89d6324 from datanode 7cdd7f1c-cc78-423f-8148-0e2bf2cf6cf9 2023-08-06 09:53:14,107 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x797192cc284db483: from storage DS-47ea95e7-b60e-4e13-a055-e60cc89d6324 node DatanodeRegistration(127.0.0.1:43573, datanodeUuid=7cdd7f1c-cc78-423f-8148-0e2bf2cf6cf9, infoPort=40399, infoSecurePort=0, ipcPort=36789, storageInfo=lv=-57;cid=testClusterID;nsid=1360210479;c=1691315591457), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-06 09:53:14,107 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4f7e8d476b04a77b: Processing first storage report for DS-ec35121d-9a7b-4f64-a436-1e62274836fa from datanode d309b271-e4de-4313-87b3-7ace32969fdc 2023-08-06 09:53:14,107 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4f7e8d476b04a77b: from storage DS-ec35121d-9a7b-4f64-a436-1e62274836fa node DatanodeRegistration(127.0.0.1:43019, datanodeUuid=d309b271-e4de-4313-87b3-7ace32969fdc, infoPort=37633, infoSecurePort=0, ipcPort=38125, storageInfo=lv=-57;cid=testClusterID;nsid=1360210479;c=1691315591457), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-06 09:53:14,108 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x797192cc284db483: Processing first storage report for DS-c8d20f90-029f-4feb-9864-7a635d5be122 from datanode 7cdd7f1c-cc78-423f-8148-0e2bf2cf6cf9 2023-08-06 09:53:14,108 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x797192cc284db483: from storage DS-c8d20f90-029f-4feb-9864-7a635d5be122 node DatanodeRegistration(127.0.0.1:43573, datanodeUuid=7cdd7f1c-cc78-423f-8148-0e2bf2cf6cf9, infoPort=40399, infoSecurePort=0, ipcPort=36789, storageInfo=lv=-57;cid=testClusterID;nsid=1360210479;c=1691315591457), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-06 09:53:14,108 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xefbdcbaab02fe3be: Processing first storage report for DS-cba4c949-ec4e-4382-8c9c-d3bb2b6f0c93 from datanode 66ee7809-9841-425b-bf64-0316ea0c97c6 2023-08-06 09:53:14,108 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xefbdcbaab02fe3be: from storage DS-cba4c949-ec4e-4382-8c9c-d3bb2b6f0c93 node DatanodeRegistration(127.0.0.1:44229, datanodeUuid=66ee7809-9841-425b-bf64-0316ea0c97c6, infoPort=39943, infoSecurePort=0, ipcPort=38769, storageInfo=lv=-57;cid=testClusterID;nsid=1360210479;c=1691315591457), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-06 09:53:14,352 DEBUG [Listener at localhost/36789] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f 2023-08-06 09:53:14,443 INFO [Listener at localhost/36789] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/zookeeper_0, clientPort=52496, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-06 09:53:14,461 INFO [Listener at localhost/36789] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=52496 2023-08-06 09:53:14,473 INFO [Listener at localhost/36789] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-06 09:53:14,476 INFO [Listener at localhost/36789] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-06 09:53:15,159 INFO [Listener at localhost/36789] util.FSUtils(471): Created version file at hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386 with version=8 2023-08-06 09:53:15,159 INFO [Listener at localhost/36789] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/hbase-staging 2023-08-06 09:53:15,521 INFO [Listener at localhost/36789] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-08-06 09:53:16,036 INFO [Listener at localhost/36789] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-06 09:53:16,069 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,070 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,070 INFO [Listener at localhost/36789] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-06 09:53:16,070 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,071 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-06 09:53:16,219 INFO [Listener at localhost/36789] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-06 09:53:16,290 DEBUG [Listener at localhost/36789] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-08-06 09:53:16,381 INFO [Listener at localhost/36789] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33207 2023-08-06 09:53:16,391 INFO [Listener at localhost/36789] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-06 09:53:16,393 INFO [Listener at localhost/36789] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-06 09:53:16,414 INFO [Listener at localhost/36789] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33207 connecting to ZooKeeper ensemble=127.0.0.1:52496 2023-08-06 09:53:16,454 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:332070x0, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-06 09:53:16,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33207-0x101d9b267bc0000 connected 2023-08-06 09:53:16,490 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-06 09:53:16,491 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-06 09:53:16,494 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-06 09:53:16,502 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33207 2023-08-06 09:53:16,502 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33207 2023-08-06 09:53:16,503 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33207 2023-08-06 09:53:16,503 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33207 2023-08-06 09:53:16,503 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33207 2023-08-06 09:53:16,509 INFO [Listener at localhost/36789] master.HMaster(444): hbase.rootdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386, hbase.cluster.distributed=false 2023-08-06 09:53:16,579 INFO [Listener at localhost/36789] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-06 09:53:16,579 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,579 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,579 INFO [Listener at localhost/36789] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-06 09:53:16,579 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,580 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-06 09:53:16,585 INFO [Listener at localhost/36789] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-06 09:53:16,588 INFO [Listener at localhost/36789] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39585 2023-08-06 09:53:16,590 INFO [Listener at localhost/36789] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-06 09:53:16,597 DEBUG [Listener at localhost/36789] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-06 09:53:16,598 INFO [Listener at localhost/36789] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-06 09:53:16,600 INFO [Listener at localhost/36789] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-06 09:53:16,602 INFO [Listener at localhost/36789] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39585 connecting to ZooKeeper ensemble=127.0.0.1:52496 2023-08-06 09:53:16,608 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:395850x0, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-06 09:53:16,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:39585-0x101d9b267bc0001 connected 2023-08-06 09:53:16,609 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-06 09:53:16,610 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-06 09:53:16,611 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-06 09:53:16,612 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39585 2023-08-06 09:53:16,612 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39585 2023-08-06 09:53:16,613 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39585 2023-08-06 09:53:16,615 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39585 2023-08-06 09:53:16,618 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39585 2023-08-06 09:53:16,631 INFO [Listener at localhost/36789] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-06 09:53:16,631 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,631 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,632 INFO [Listener at localhost/36789] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-06 09:53:16,632 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,632 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-06 09:53:16,632 INFO [Listener at localhost/36789] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-06 09:53:16,633 INFO [Listener at localhost/36789] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44015 2023-08-06 09:53:16,634 INFO [Listener at localhost/36789] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-06 09:53:16,638 DEBUG [Listener at localhost/36789] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-06 09:53:16,639 INFO [Listener at localhost/36789] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-06 09:53:16,642 INFO [Listener at localhost/36789] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-06 09:53:16,644 INFO [Listener at localhost/36789] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44015 connecting to ZooKeeper ensemble=127.0.0.1:52496 2023-08-06 09:53:16,648 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:440150x0, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-06 09:53:16,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:44015-0x101d9b267bc0002 connected 2023-08-06 09:53:16,649 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-06 09:53:16,650 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-06 09:53:16,651 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-06 09:53:16,655 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44015 2023-08-06 09:53:16,655 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44015 2023-08-06 09:53:16,656 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44015 2023-08-06 09:53:16,656 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44015 2023-08-06 09:53:16,657 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44015 2023-08-06 09:53:16,669 INFO [Listener at localhost/36789] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-06 09:53:16,669 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,670 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,670 INFO [Listener at localhost/36789] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-06 09:53:16,670 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-06 09:53:16,670 INFO [Listener at localhost/36789] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-06 09:53:16,670 INFO [Listener at localhost/36789] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-06 09:53:16,672 INFO [Listener at localhost/36789] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40847 2023-08-06 09:53:16,673 INFO [Listener at localhost/36789] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-06 09:53:16,674 DEBUG [Listener at localhost/36789] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-06 09:53:16,675 INFO [Listener at localhost/36789] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-06 09:53:16,677 INFO [Listener at localhost/36789] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-06 09:53:16,678 INFO [Listener at localhost/36789] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40847 connecting to ZooKeeper ensemble=127.0.0.1:52496 2023-08-06 09:53:16,683 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:408470x0, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-06 09:53:16,685 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-06 09:53:16,685 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40847-0x101d9b267bc0003 connected 2023-08-06 09:53:16,685 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-06 09:53:16,686 DEBUG [Listener at localhost/36789] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-06 09:53:16,691 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40847 2023-08-06 09:53:16,691 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40847 2023-08-06 09:53:16,691 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40847 2023-08-06 09:53:16,692 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40847 2023-08-06 09:53:16,692 DEBUG [Listener at localhost/36789] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40847 2023-08-06 09:53:16,695 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33207,1691315595337 2023-08-06 09:53:16,705 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-06 09:53:16,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33207,1691315595337 2023-08-06 09:53:16,726 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-06 09:53:16,726 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-06 09:53:16,726 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-06 09:53:16,726 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-06 09:53:16,726 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-06 09:53:16,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-06 09:53:16,730 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33207,1691315595337 from backup master directory 2023-08-06 09:53:16,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-06 09:53:16,734 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33207,1691315595337 2023-08-06 09:53:16,734 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-06 09:53:16,735 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-06 09:53:16,735 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33207,1691315595337 2023-08-06 09:53:16,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-08-06 09:53:16,739 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-08-06 09:53:16,828 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/hbase.id with ID: 050e1556-bc73-4ff6-94ed-ef0840369805 2023-08-06 09:53:16,868 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-06 09:53:16,883 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-06 09:53:16,954 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x59654f68 to 127.0.0.1:52496 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-06 09:53:16,987 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@48fb7061, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-06 09:53:17,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-06 09:53:17,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-06 09:53:17,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-08-06 09:53:17,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-08-06 09:53:17,032 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-06 09:53:17,036 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-06 09:53:17,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-06 09:53:17,073 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/data/master/store-tmp 2023-08-06 09:53:17,111 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:17,111 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-06 09:53:17,111 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-06 09:53:17,111 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-06 09:53:17,111 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-06 09:53:17,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-06 09:53:17,112 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-06 09:53:17,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-06 09:53:17,113 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/WALs/jenkins-hbase4.apache.org,33207,1691315595337 2023-08-06 09:53:17,141 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33207%2C1691315595337, suffix=, logDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/WALs/jenkins-hbase4.apache.org,33207,1691315595337, archiveDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/oldWALs, maxLogs=10 2023-08-06 09:53:17,208 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43573,DS-47ea95e7-b60e-4e13-a055-e60cc89d6324,DISK] 2023-08-06 09:53:17,208 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44229,DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4,DISK] 2023-08-06 09:53:17,208 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43019,DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d,DISK] 2023-08-06 09:53:17,216 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-06 09:53:17,287 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/WALs/jenkins-hbase4.apache.org,33207,1691315595337/jenkins-hbase4.apache.org%2C33207%2C1691315595337.1691315597156 2023-08-06 09:53:17,287 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44229,DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4,DISK], DatanodeInfoWithStorage[127.0.0.1:43019,DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d,DISK], DatanodeInfoWithStorage[127.0.0.1:43573,DS-47ea95e7-b60e-4e13-a055-e60cc89d6324,DISK]] 2023-08-06 09:53:17,288 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:53:17,288 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:17,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-06 09:53:17,293 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-06 09:53:17,350 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-06 09:53:17,358 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-06 09:53:17,385 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-06 09:53:17,399 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:17,405 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-06 09:53:17,407 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-06 09:53:17,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-06 09:53:17,427 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:53:17,428 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72520916, jitterRate=0.0806458592414856}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-06 09:53:17,429 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-06 09:53:17,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-06 09:53:17,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:17,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:17,452 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:17,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-08-06 09:53:17,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-08-06 09:53:17,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-06 09:53:17,511 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-06 09:53:17,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-06 09:53:17,546 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-06 09:53:17,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-06 09:53:17,553 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-06 09:53:17,560 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-06 09:53:17,566 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-06 09:53:17,569 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-06 09:53:17,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-06 09:53:17,571 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-06 09:53:17,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-06 09:53:17,588 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-06 09:53:17,588 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-06 09:53:17,588 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-06 09:53:17,588 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-06 09:53:17,588 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-06 09:53:17,589 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33207,1691315595337, sessionid=0x101d9b267bc0000, setting cluster-up flag (Was=false) 2023-08-06 09:53:17,614 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-06 09:53:17,619 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-06 09:53:17,620 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33207,1691315595337 2023-08-06 09:53:17,626 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-06 09:53:17,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-06 09:53:17,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33207,1691315595337 2023-08-06 09:53:17,638 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.hbase-snapshot/.tmp 2023-08-06 09:53:17,698 INFO [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer(951): ClusterId : 050e1556-bc73-4ff6-94ed-ef0840369805 2023-08-06 09:53:17,698 INFO [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(951): ClusterId : 050e1556-bc73-4ff6-94ed-ef0840369805 2023-08-06 09:53:17,698 INFO [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(951): ClusterId : 050e1556-bc73-4ff6-94ed-ef0840369805 2023-08-06 09:53:17,706 DEBUG [RS:2;jenkins-hbase4:40847] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-06 09:53:17,706 DEBUG [RS:1;jenkins-hbase4:44015] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-06 09:53:17,706 DEBUG [RS:0;jenkins-hbase4:39585] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-06 09:53:17,718 DEBUG [RS:0;jenkins-hbase4:39585] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-06 09:53:17,718 DEBUG [RS:2;jenkins-hbase4:40847] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-06 09:53:17,718 DEBUG [RS:1;jenkins-hbase4:44015] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-06 09:53:17,718 DEBUG [RS:2;jenkins-hbase4:40847] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-06 09:53:17,718 DEBUG [RS:0;jenkins-hbase4:39585] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-06 09:53:17,718 DEBUG [RS:1;jenkins-hbase4:44015] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-06 09:53:17,723 DEBUG [RS:1;jenkins-hbase4:44015] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-06 09:53:17,723 DEBUG [RS:0;jenkins-hbase4:39585] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-06 09:53:17,723 DEBUG [RS:2;jenkins-hbase4:40847] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-06 09:53:17,725 DEBUG [RS:1;jenkins-hbase4:44015] zookeeper.ReadOnlyZKClient(139): Connect 0x3cb02621 to 127.0.0.1:52496 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-06 09:53:17,726 DEBUG [RS:0;jenkins-hbase4:39585] zookeeper.ReadOnlyZKClient(139): Connect 0x20097838 to 127.0.0.1:52496 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-06 09:53:17,726 DEBUG [RS:2;jenkins-hbase4:40847] zookeeper.ReadOnlyZKClient(139): Connect 0x70a3177b to 127.0.0.1:52496 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-06 09:53:17,744 DEBUG [RS:1;jenkins-hbase4:44015] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3277dfc7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-06 09:53:17,744 DEBUG [RS:2;jenkins-hbase4:40847] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e64e71f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-06 09:53:17,744 DEBUG [RS:1;jenkins-hbase4:44015] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36484d51, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-06 09:53:17,744 DEBUG [RS:2;jenkins-hbase4:40847] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7a5b7c0b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-06 09:53:17,745 DEBUG [RS:0;jenkins-hbase4:39585] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2ece9249, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-06 09:53:17,745 DEBUG [RS:0;jenkins-hbase4:39585] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2af11348, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-06 09:53:17,774 DEBUG [RS:0;jenkins-hbase4:39585] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:39585 2023-08-06 09:53:17,776 DEBUG [RS:2;jenkins-hbase4:40847] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:40847 2023-08-06 09:53:17,779 DEBUG [RS:1;jenkins-hbase4:44015] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:44015 2023-08-06 09:53:17,781 INFO [RS:2;jenkins-hbase4:40847] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-06 09:53:17,781 INFO [RS:1;jenkins-hbase4:44015] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-06 09:53:17,781 INFO [RS:1;jenkins-hbase4:44015] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-06 09:53:17,781 INFO [RS:0;jenkins-hbase4:39585] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-06 09:53:17,781 DEBUG [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1022): About to register with Master. 2023-08-06 09:53:17,781 INFO [RS:2;jenkins-hbase4:40847] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-06 09:53:17,781 INFO [RS:0;jenkins-hbase4:39585] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-06 09:53:17,782 DEBUG [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer(1022): About to register with Master. 2023-08-06 09:53:17,782 DEBUG [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1022): About to register with Master. 2023-08-06 09:53:17,784 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:17,785 INFO [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33207,1691315595337 with isa=jenkins-hbase4.apache.org/172.31.14.131:40847, startcode=1691315596668 2023-08-06 09:53:17,785 INFO [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33207,1691315595337 with isa=jenkins-hbase4.apache.org/172.31.14.131:44015, startcode=1691315596630 2023-08-06 09:53:17,785 INFO [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33207,1691315595337 with isa=jenkins-hbase4.apache.org/172.31.14.131:39585, startcode=1691315596578 2023-08-06 09:53:17,795 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-06 09:53:17,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-06 09:53:17,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-06 09:53:17,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-06 09:53:17,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-06 09:53:17,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:17,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-06 09:53:17,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:17,797 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315627797 2023-08-06 09:53:17,800 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-06 09:53:17,804 DEBUG [RS:1;jenkins-hbase4:44015] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-06 09:53:17,804 DEBUG [RS:2;jenkins-hbase4:40847] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-06 09:53:17,804 DEBUG [RS:0;jenkins-hbase4:39585] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-06 09:53:17,806 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:17,807 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-06 09:53:17,813 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-06 09:53:17,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-06 09:53:17,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-06 09:53:17,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-06 09:53:17,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-06 09:53:17,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-06 09:53:17,823 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:17,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-06 09:53:17,827 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-06 09:53:17,827 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-06 09:53:17,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-06 09:53:17,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-06 09:53:17,834 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691315597834,5,FailOnTimeoutGroup] 2023-08-06 09:53:17,839 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691315597835,5,FailOnTimeoutGroup] 2023-08-06 09:53:17,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:17,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-06 09:53:17,841 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:17,841 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:17,867 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58657, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-08-06 09:53:17,867 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34557, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-08-06 09:53:17,867 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36489, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-08-06 09:53:17,878 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-06 09:53:17,879 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-06 09:53:17,879 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386 2023-08-06 09:53:17,881 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33207] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:17,897 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33207] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:17,897 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33207] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:17,910 DEBUG [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386 2023-08-06 09:53:17,910 DEBUG [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386 2023-08-06 09:53:17,911 DEBUG [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:43345 2023-08-06 09:53:17,911 DEBUG [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-06 09:53:17,910 DEBUG [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386 2023-08-06 09:53:17,912 DEBUG [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:43345 2023-08-06 09:53:17,912 DEBUG [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-06 09:53:17,910 DEBUG [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:43345 2023-08-06 09:53:17,912 DEBUG [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-06 09:53:17,917 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:17,919 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-06 09:53:17,923 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/info 2023-08-06 09:53:17,923 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-06 09:53:17,923 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-06 09:53:17,924 DEBUG [RS:1;jenkins-hbase4:44015] zookeeper.ZKUtil(162): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:17,924 WARN [RS:1;jenkins-hbase4:44015] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-06 09:53:17,924 DEBUG [RS:0;jenkins-hbase4:39585] zookeeper.ZKUtil(162): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:17,925 INFO [RS:1;jenkins-hbase4:44015] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-06 09:53:17,925 WARN [RS:0;jenkins-hbase4:39585] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-06 09:53:17,925 DEBUG [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1948): logDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/WALs/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:17,925 INFO [RS:0;jenkins-hbase4:39585] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-06 09:53:17,925 DEBUG [RS:2;jenkins-hbase4:40847] zookeeper.ZKUtil(162): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:17,926 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:17,926 WARN [RS:2;jenkins-hbase4:40847] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-06 09:53:17,926 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40847,1691315596668] 2023-08-06 09:53:17,926 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-06 09:53:17,926 DEBUG [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer(1948): logDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/WALs/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:17,926 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39585,1691315596578] 2023-08-06 09:53:17,926 INFO [RS:2;jenkins-hbase4:40847] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-06 09:53:17,926 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,44015,1691315596630] 2023-08-06 09:53:17,927 DEBUG [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1948): logDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/WALs/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:17,933 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/rep_barrier 2023-08-06 09:53:17,934 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-06 09:53:17,935 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:17,935 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-06 09:53:17,937 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/table 2023-08-06 09:53:17,938 DEBUG [RS:0;jenkins-hbase4:39585] zookeeper.ZKUtil(162): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:17,938 DEBUG [RS:1;jenkins-hbase4:44015] zookeeper.ZKUtil(162): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:17,938 DEBUG [RS:2;jenkins-hbase4:40847] zookeeper.ZKUtil(162): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:17,939 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-06 09:53:17,940 DEBUG [RS:0;jenkins-hbase4:39585] zookeeper.ZKUtil(162): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:17,940 DEBUG [RS:2;jenkins-hbase4:40847] zookeeper.ZKUtil(162): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:17,940 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:17,940 DEBUG [RS:1;jenkins-hbase4:44015] zookeeper.ZKUtil(162): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:17,942 DEBUG [RS:1;jenkins-hbase4:44015] zookeeper.ZKUtil(162): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:17,942 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740 2023-08-06 09:53:17,942 DEBUG [RS:2;jenkins-hbase4:40847] zookeeper.ZKUtil(162): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:17,943 DEBUG [RS:0;jenkins-hbase4:39585] zookeeper.ZKUtil(162): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:17,943 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740 2023-08-06 09:53:17,948 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-06 09:53:17,951 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-06 09:53:17,954 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:53:17,955 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59684741, jitterRate=-0.11062805354595184}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-06 09:53:17,955 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-06 09:53:17,955 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-06 09:53:17,955 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-06 09:53:17,955 DEBUG [RS:0;jenkins-hbase4:39585] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-06 09:53:17,955 DEBUG [RS:1;jenkins-hbase4:44015] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-06 09:53:17,955 DEBUG [RS:2;jenkins-hbase4:40847] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-06 09:53:17,955 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-06 09:53:17,956 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-06 09:53:17,956 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-06 09:53:17,958 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-06 09:53:17,958 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-06 09:53:17,965 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:17,965 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-06 09:53:17,967 INFO [RS:0;jenkins-hbase4:39585] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-06 09:53:17,968 INFO [RS:2;jenkins-hbase4:40847] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-06 09:53:17,967 INFO [RS:1;jenkins-hbase4:44015] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-06 09:53:17,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-06 09:53:17,993 INFO [RS:2;jenkins-hbase4:40847] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-06 09:53:17,995 INFO [RS:0;jenkins-hbase4:39585] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-06 09:53:17,995 INFO [RS:1;jenkins-hbase4:44015] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-06 09:53:17,998 INFO [RS:2;jenkins-hbase4:40847] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-06 09:53:17,998 INFO [RS:2;jenkins-hbase4:40847] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:17,999 INFO [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-06 09:53:17,999 INFO [RS:0;jenkins-hbase4:39585] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-06 09:53:17,999 INFO [RS:1;jenkins-hbase4:44015] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-06 09:53:17,999 INFO [RS:0;jenkins-hbase4:39585] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,000 INFO [RS:1;jenkins-hbase4:44015] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,001 INFO [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-06 09:53:18,001 INFO [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-06 09:53:18,005 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-06 09:53:18,007 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-06 09:53:18,011 INFO [RS:0;jenkins-hbase4:39585] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,011 INFO [RS:1;jenkins-hbase4:44015] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,011 INFO [RS:2;jenkins-hbase4:40847] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,011 DEBUG [RS:0;jenkins-hbase4:39585] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,011 DEBUG [RS:1;jenkins-hbase4:44015] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,012 DEBUG [RS:0;jenkins-hbase4:39585] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,012 DEBUG [RS:2;jenkins-hbase4:40847] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,012 DEBUG [RS:0;jenkins-hbase4:39585] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,012 DEBUG [RS:2;jenkins-hbase4:40847] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,012 DEBUG [RS:0;jenkins-hbase4:39585] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,012 DEBUG [RS:2;jenkins-hbase4:40847] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,012 DEBUG [RS:0;jenkins-hbase4:39585] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,012 DEBUG [RS:2;jenkins-hbase4:40847] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:0;jenkins-hbase4:39585] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-06 09:53:18,013 DEBUG [RS:2;jenkins-hbase4:40847] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:0;jenkins-hbase4:39585] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:2;jenkins-hbase4:40847] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-06 09:53:18,012 DEBUG [RS:1;jenkins-hbase4:44015] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:2;jenkins-hbase4:40847] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:0;jenkins-hbase4:39585] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:1;jenkins-hbase4:44015] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:0;jenkins-hbase4:39585] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:2;jenkins-hbase4:40847] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:0;jenkins-hbase4:39585] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:1;jenkins-hbase4:44015] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:2;jenkins-hbase4:40847] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:1;jenkins-hbase4:44015] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,013 DEBUG [RS:2;jenkins-hbase4:40847] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,014 DEBUG [RS:1;jenkins-hbase4:44015] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-06 09:53:18,014 DEBUG [RS:1;jenkins-hbase4:44015] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,014 DEBUG [RS:1;jenkins-hbase4:44015] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,014 DEBUG [RS:1;jenkins-hbase4:44015] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,014 DEBUG [RS:1;jenkins-hbase4:44015] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-06 09:53:18,018 INFO [RS:2;jenkins-hbase4:40847] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,019 INFO [RS:2;jenkins-hbase4:40847] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,019 INFO [RS:2;jenkins-hbase4:40847] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,022 INFO [RS:1;jenkins-hbase4:44015] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,023 INFO [RS:1;jenkins-hbase4:44015] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,023 INFO [RS:1;jenkins-hbase4:44015] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,023 INFO [RS:0;jenkins-hbase4:39585] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,023 INFO [RS:0;jenkins-hbase4:39585] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,023 INFO [RS:0;jenkins-hbase4:39585] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,039 INFO [RS:2;jenkins-hbase4:40847] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-06 09:53:18,039 INFO [RS:1;jenkins-hbase4:44015] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-06 09:53:18,045 INFO [RS:0;jenkins-hbase4:39585] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-06 09:53:18,045 INFO [RS:2;jenkins-hbase4:40847] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40847,1691315596668-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,045 INFO [RS:0;jenkins-hbase4:39585] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39585,1691315596578-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,045 INFO [RS:1;jenkins-hbase4:44015] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44015,1691315596630-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,064 INFO [RS:0;jenkins-hbase4:39585] regionserver.Replication(203): jenkins-hbase4.apache.org,39585,1691315596578 started 2023-08-06 09:53:18,064 INFO [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,39585,1691315596578, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39585, sessionid=0x101d9b267bc0001 2023-08-06 09:53:18,064 DEBUG [RS:0;jenkins-hbase4:39585] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-06 09:53:18,065 DEBUG [RS:0;jenkins-hbase4:39585] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:18,065 DEBUG [RS:0;jenkins-hbase4:39585] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39585,1691315596578' 2023-08-06 09:53:18,065 DEBUG [RS:0;jenkins-hbase4:39585] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:18,065 DEBUG [RS:0;jenkins-hbase4:39585] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:18,066 DEBUG [RS:0;jenkins-hbase4:39585] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-06 09:53:18,066 DEBUG [RS:0;jenkins-hbase4:39585] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-06 09:53:18,066 DEBUG [RS:0;jenkins-hbase4:39585] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:18,066 DEBUG [RS:0;jenkins-hbase4:39585] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39585,1691315596578' 2023-08-06 09:53:18,066 DEBUG [RS:0;jenkins-hbase4:39585] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-06 09:53:18,067 DEBUG [RS:0;jenkins-hbase4:39585] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-06 09:53:18,067 INFO [RS:1;jenkins-hbase4:44015] regionserver.Replication(203): jenkins-hbase4.apache.org,44015,1691315596630 started 2023-08-06 09:53:18,067 DEBUG [RS:0;jenkins-hbase4:39585] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-06 09:53:18,067 INFO [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,44015,1691315596630, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:44015, sessionid=0x101d9b267bc0002 2023-08-06 09:53:18,067 INFO [RS:2;jenkins-hbase4:40847] regionserver.Replication(203): jenkins-hbase4.apache.org,40847,1691315596668 started 2023-08-06 09:53:18,067 DEBUG [RS:1;jenkins-hbase4:44015] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-06 09:53:18,067 INFO [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,40847,1691315596668, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40847, sessionid=0x101d9b267bc0003 2023-08-06 09:53:18,067 INFO [RS:0;jenkins-hbase4:39585] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-06 09:53:18,068 DEBUG [RS:2;jenkins-hbase4:40847] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-06 09:53:18,068 INFO [RS:0;jenkins-hbase4:39585] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-06 09:53:18,068 DEBUG [RS:1;jenkins-hbase4:44015] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:18,068 DEBUG [RS:2;jenkins-hbase4:40847] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:18,069 DEBUG [RS:2;jenkins-hbase4:40847] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40847,1691315596668' 2023-08-06 09:53:18,069 DEBUG [RS:1;jenkins-hbase4:44015] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44015,1691315596630' 2023-08-06 09:53:18,069 DEBUG [RS:2;jenkins-hbase4:40847] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:18,069 DEBUG [RS:1;jenkins-hbase4:44015] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:18,070 DEBUG [RS:2;jenkins-hbase4:40847] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:18,070 DEBUG [RS:1;jenkins-hbase4:44015] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:18,070 DEBUG [RS:2;jenkins-hbase4:40847] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-06 09:53:18,070 DEBUG [RS:2;jenkins-hbase4:40847] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-06 09:53:18,070 DEBUG [RS:2;jenkins-hbase4:40847] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:18,071 DEBUG [RS:1;jenkins-hbase4:44015] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-06 09:53:18,071 DEBUG [RS:2;jenkins-hbase4:40847] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40847,1691315596668' 2023-08-06 09:53:18,071 DEBUG [RS:2;jenkins-hbase4:40847] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-06 09:53:18,071 DEBUG [RS:1;jenkins-hbase4:44015] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-06 09:53:18,071 DEBUG [RS:1;jenkins-hbase4:44015] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:18,071 DEBUG [RS:1;jenkins-hbase4:44015] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44015,1691315596630' 2023-08-06 09:53:18,072 DEBUG [RS:1;jenkins-hbase4:44015] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-06 09:53:18,075 DEBUG [RS:2;jenkins-hbase4:40847] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-06 09:53:18,075 DEBUG [RS:1;jenkins-hbase4:44015] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-06 09:53:18,075 DEBUG [RS:2;jenkins-hbase4:40847] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-06 09:53:18,076 INFO [RS:2;jenkins-hbase4:40847] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-06 09:53:18,076 DEBUG [RS:1;jenkins-hbase4:44015] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-06 09:53:18,076 INFO [RS:2;jenkins-hbase4:40847] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-06 09:53:18,076 INFO [RS:1;jenkins-hbase4:44015] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-06 09:53:18,076 INFO [RS:1;jenkins-hbase4:44015] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-06 09:53:18,159 DEBUG [jenkins-hbase4:33207] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-08-06 09:53:18,162 DEBUG [jenkins-hbase4:33207] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:53:18,168 DEBUG [jenkins-hbase4:33207] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:53:18,168 DEBUG [jenkins-hbase4:33207] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:53:18,168 DEBUG [jenkins-hbase4:33207] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:53:18,168 DEBUG [jenkins-hbase4:33207] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:53:18,172 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44015,1691315596630, state=OPENING 2023-08-06 09:53:18,180 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-06 09:53:18,181 INFO [RS:0;jenkins-hbase4:39585] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39585%2C1691315596578, suffix=, logDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/WALs/jenkins-hbase4.apache.org,39585,1691315596578, archiveDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/oldWALs, maxLogs=32 2023-08-06 09:53:18,181 INFO [RS:2;jenkins-hbase4:40847] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40847%2C1691315596668, suffix=, logDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/WALs/jenkins-hbase4.apache.org,40847,1691315596668, archiveDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/oldWALs, maxLogs=32 2023-08-06 09:53:18,182 INFO [RS:1;jenkins-hbase4:44015] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44015%2C1691315596630, suffix=, logDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/WALs/jenkins-hbase4.apache.org,44015,1691315596630, archiveDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/oldWALs, maxLogs=32 2023-08-06 09:53:18,183 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-06 09:53:18,184 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-06 09:53:18,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:18,210 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44229,DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4,DISK] 2023-08-06 09:53:18,210 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43019,DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d,DISK] 2023-08-06 09:53:18,210 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43573,DS-47ea95e7-b60e-4e13-a055-e60cc89d6324,DISK] 2023-08-06 09:53:18,232 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44229,DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4,DISK] 2023-08-06 09:53:18,232 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43573,DS-47ea95e7-b60e-4e13-a055-e60cc89d6324,DISK] 2023-08-06 09:53:18,232 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43019,DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d,DISK] 2023-08-06 09:53:18,234 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43573,DS-47ea95e7-b60e-4e13-a055-e60cc89d6324,DISK] 2023-08-06 09:53:18,234 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43019,DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d,DISK] 2023-08-06 09:53:18,235 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44229,DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4,DISK] 2023-08-06 09:53:18,235 INFO [RS:0;jenkins-hbase4:39585] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/WALs/jenkins-hbase4.apache.org,39585,1691315596578/jenkins-hbase4.apache.org%2C39585%2C1691315596578.1691315598186 2023-08-06 09:53:18,246 DEBUG [RS:0;jenkins-hbase4:39585] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44229,DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4,DISK], DatanodeInfoWithStorage[127.0.0.1:43019,DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d,DISK], DatanodeInfoWithStorage[127.0.0.1:43573,DS-47ea95e7-b60e-4e13-a055-e60cc89d6324,DISK]] 2023-08-06 09:53:18,247 INFO [RS:2;jenkins-hbase4:40847] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/WALs/jenkins-hbase4.apache.org,40847,1691315596668/jenkins-hbase4.apache.org%2C40847%2C1691315596668.1691315598186 2023-08-06 09:53:18,251 DEBUG [RS:2;jenkins-hbase4:40847] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44229,DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4,DISK], DatanodeInfoWithStorage[127.0.0.1:43019,DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d,DISK], DatanodeInfoWithStorage[127.0.0.1:43573,DS-47ea95e7-b60e-4e13-a055-e60cc89d6324,DISK]] 2023-08-06 09:53:18,254 INFO [RS:1;jenkins-hbase4:44015] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/WALs/jenkins-hbase4.apache.org,44015,1691315596630/jenkins-hbase4.apache.org%2C44015%2C1691315596630.1691315598186 2023-08-06 09:53:18,254 DEBUG [RS:1;jenkins-hbase4:44015] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43019,DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d,DISK], DatanodeInfoWithStorage[127.0.0.1:44229,DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4,DISK], DatanodeInfoWithStorage[127.0.0.1:43573,DS-47ea95e7-b60e-4e13-a055-e60cc89d6324,DISK]] 2023-08-06 09:53:18,385 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:18,388 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-06 09:53:18,391 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54722, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-06 09:53:18,403 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-06 09:53:18,404 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-06 09:53:18,407 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44015%2C1691315596630.meta, suffix=.meta, logDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/WALs/jenkins-hbase4.apache.org,44015,1691315596630, archiveDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/oldWALs, maxLogs=32 2023-08-06 09:53:18,428 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43573,DS-47ea95e7-b60e-4e13-a055-e60cc89d6324,DISK] 2023-08-06 09:53:18,429 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43019,DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d,DISK] 2023-08-06 09:53:18,429 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44229,DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4,DISK] 2023-08-06 09:53:18,436 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/WALs/jenkins-hbase4.apache.org,44015,1691315596630/jenkins-hbase4.apache.org%2C44015%2C1691315596630.meta.1691315598408.meta 2023-08-06 09:53:18,437 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43573,DS-47ea95e7-b60e-4e13-a055-e60cc89d6324,DISK], DatanodeInfoWithStorage[127.0.0.1:43019,DS-169b7a23-dbb2-4e11-81da-0eab2798cc3d,DISK], DatanodeInfoWithStorage[127.0.0.1:44229,DS-8082c173-eaa2-4cbc-ac7e-ae2cff7018d4,DISK]] 2023-08-06 09:53:18,437 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:53:18,441 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:18,442 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-06 09:53:18,457 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-06 09:53:18,460 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-06 09:53:18,464 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-06 09:53:18,464 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:18,464 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-06 09:53:18,465 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-06 09:53:18,468 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-06 09:53:18,471 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/info 2023-08-06 09:53:18,471 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/info 2023-08-06 09:53:18,471 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-06 09:53:18,472 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:18,472 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-06 09:53:18,474 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/rep_barrier 2023-08-06 09:53:18,474 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/rep_barrier 2023-08-06 09:53:18,475 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-06 09:53:18,477 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:18,478 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-06 09:53:18,479 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/table 2023-08-06 09:53:18,479 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/table 2023-08-06 09:53:18,480 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-06 09:53:18,481 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:18,482 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740 2023-08-06 09:53:18,485 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740 2023-08-06 09:53:18,488 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-06 09:53:18,491 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-06 09:53:18,492 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70743558, jitterRate=0.05416116118431091}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-06 09:53:18,493 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-06 09:53:18,503 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1691315598378 2023-08-06 09:53:18,520 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-06 09:53:18,521 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-06 09:53:18,521 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44015,1691315596630, state=OPEN 2023-08-06 09:53:18,524 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-06 09:53:18,524 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-06 09:53:18,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-06 09:53:18,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44015,1691315596630 in 337 msec 2023-08-06 09:53:18,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-06 09:53:18,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 553 msec 2023-08-06 09:53:18,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 857 msec 2023-08-06 09:53:18,541 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691315598541, completionTime=-1 2023-08-06 09:53:18,541 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-08-06 09:53:18,541 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:18,609 DEBUG [hconnection-0x709851dd-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-06 09:53:18,612 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54738, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-06 09:53:18,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:18,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315658628 2023-08-06 09:53:18,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315718628 2023-08-06 09:53:18,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 86 msec 2023-08-06 09:53:18,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33207,1691315595337-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33207,1691315595337-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,651 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33207,1691315595337-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,652 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33207, period=300000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,653 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-06 09:53:18,663 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-06 09:53:18,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-06 09:53:18,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-06 09:53:18,682 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:18,686 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-06 09:53:18,689 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-06 09:53:18,713 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964 2023-08-06 09:53:18,715 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964 empty. 2023-08-06 09:53:18,716 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964 2023-08-06 09:53:18,716 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-06 09:53:18,751 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-06 09:53:18,753 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => ebbcd508c2d5b7b6263f7d0763491964, NAME => 'hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp 2023-08-06 09:53:18,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:18,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing ebbcd508c2d5b7b6263f7d0763491964, disabling compactions & flushes 2023-08-06 09:53:18,770 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:53:18,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:53:18,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. after waiting 0 ms 2023-08-06 09:53:18,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:53:18,770 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:53:18,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for ebbcd508c2d5b7b6263f7d0763491964: 2023-08-06 09:53:18,775 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-06 09:53:18,791 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691315598777"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315598777"}]},"ts":"1691315598777"} 2023-08-06 09:53:18,818 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-06 09:53:18,820 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-06 09:53:18,824 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315598820"}]},"ts":"1691315598820"} 2023-08-06 09:53:18,828 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-06 09:53:18,834 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:53:18,835 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:53:18,835 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:53:18,835 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:53:18,835 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:53:18,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ebbcd508c2d5b7b6263f7d0763491964, ASSIGN}] 2023-08-06 09:53:18,841 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ebbcd508c2d5b7b6263f7d0763491964, ASSIGN 2023-08-06 09:53:18,843 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ebbcd508c2d5b7b6263f7d0763491964, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40847,1691315596668; forceNewPlan=false, retain=false 2023-08-06 09:53:18,995 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:53:18,996 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:18,997 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691315598996"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315598996"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315598996"}]},"ts":"1691315598996"} 2023-08-06 09:53:19,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure ebbcd508c2d5b7b6263f7d0763491964, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:53:19,157 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:19,158 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-06 09:53:19,162 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57804, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-06 09:53:19,168 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:53:19,169 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ebbcd508c2d5b7b6263f7d0763491964, NAME => 'hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:53:19,170 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:19,171 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ebbcd508c2d5b7b6263f7d0763491964 2023-08-06 09:53:19,171 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:19,171 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ebbcd508c2d5b7b6263f7d0763491964 2023-08-06 09:53:19,171 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ebbcd508c2d5b7b6263f7d0763491964 2023-08-06 09:53:19,173 INFO [StoreOpener-ebbcd508c2d5b7b6263f7d0763491964-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ebbcd508c2d5b7b6263f7d0763491964 2023-08-06 09:53:19,176 DEBUG [StoreOpener-ebbcd508c2d5b7b6263f7d0763491964-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964/info 2023-08-06 09:53:19,176 DEBUG [StoreOpener-ebbcd508c2d5b7b6263f7d0763491964-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964/info 2023-08-06 09:53:19,176 INFO [StoreOpener-ebbcd508c2d5b7b6263f7d0763491964-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ebbcd508c2d5b7b6263f7d0763491964 columnFamilyName info 2023-08-06 09:53:19,177 INFO [StoreOpener-ebbcd508c2d5b7b6263f7d0763491964-1] regionserver.HStore(310): Store=ebbcd508c2d5b7b6263f7d0763491964/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:19,179 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964 2023-08-06 09:53:19,180 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964 2023-08-06 09:53:19,184 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ebbcd508c2d5b7b6263f7d0763491964 2023-08-06 09:53:19,188 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:53:19,188 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ebbcd508c2d5b7b6263f7d0763491964; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65835086, jitterRate=-0.01898077130317688}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-06 09:53:19,188 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ebbcd508c2d5b7b6263f7d0763491964: 2023-08-06 09:53:19,190 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964., pid=6, masterSystemTime=1691315599157 2023-08-06 09:53:19,194 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:53:19,195 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:53:19,196 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:19,197 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691315599196"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315599196"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315599196"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315599196"}]},"ts":"1691315599196"} 2023-08-06 09:53:19,203 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-06 09:53:19,203 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure ebbcd508c2d5b7b6263f7d0763491964, server=jenkins-hbase4.apache.org,40847,1691315596668 in 196 msec 2023-08-06 09:53:19,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-06 09:53:19,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ebbcd508c2d5b7b6263f7d0763491964, ASSIGN in 365 msec 2023-08-06 09:53:19,208 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-06 09:53:19,208 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315599208"}]},"ts":"1691315599208"} 2023-08-06 09:53:19,211 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-06 09:53:19,214 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-06 09:53:19,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 541 msec 2023-08-06 09:53:19,285 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-06 09:53:19,287 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-06 09:53:19,287 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-06 09:53:19,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-06 09:53:19,311 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57814, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-06 09:53:19,328 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:19,348 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-06 09:53:19,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 37 msec 2023-08-06 09:53:19,361 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:19,376 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-06 09:53:19,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 19 msec 2023-08-06 09:53:19,388 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-06 09:53:19,390 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-06 09:53:19,391 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.655sec 2023-08-06 09:53:19,394 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-06 09:53:19,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-06 09:53:19,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-06 09:53:19,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33207,1691315595337-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-06 09:53:19,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33207,1691315595337-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-06 09:53:19,403 DEBUG [Listener at localhost/36789] zookeeper.ReadOnlyZKClient(139): Connect 0x69c0a999 to 127.0.0.1:52496 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-06 09:53:19,409 DEBUG [Listener at localhost/36789] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@29f4570d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-06 09:53:19,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-06 09:53:19,424 DEBUG [hconnection-0x45a8b49f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-06 09:53:19,436 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54744, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-06 09:53:19,445 INFO [Listener at localhost/36789] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,33207,1691315595337 2023-08-06 09:53:19,475 INFO [Listener at localhost/36789] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=361, OpenFileDescriptor=700, MaxFileDescriptor=60000, SystemLoadAverage=264, ProcessCount=175, AvailableMemoryMB=2542 2023-08-06 09:53:19,480 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:19,489 DEBUG [Listener at localhost/36789] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-06 09:53:19,493 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35758, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-06 09:53:19,507 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-06 09:53:19,507 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-06 09:53:19,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-06 09:53:19,513 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:19,516 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-08-06 09:53:19,517 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-06 09:53:19,520 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:19,521 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e empty. 2023-08-06 09:53:19,523 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:19,523 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-08-06 09:53:19,547 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-08-06 09:53:19,548 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => c13a71834623cd8c0cd6adbeac46b34e, NAME => 'testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp 2023-08-06 09:53:19,569 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:19,570 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing c13a71834623cd8c0cd6adbeac46b34e, disabling compactions & flushes 2023-08-06 09:53:19,570 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:19,570 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:19,570 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. after waiting 0 ms 2023-08-06 09:53:19,570 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:19,570 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:19,570 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for c13a71834623cd8c0cd6adbeac46b34e: 2023-08-06 09:53:19,574 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-08-06 09:53:19,576 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691315599575"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315599575"}]},"ts":"1691315599575"} 2023-08-06 09:53:19,578 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-06 09:53:19,580 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-06 09:53:19,580 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315599580"}]},"ts":"1691315599580"} 2023-08-06 09:53:19,582 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-08-06 09:53:19,587 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:53:19,589 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:53:19,589 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:53:19,589 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:53:19,589 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:53:19,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c13a71834623cd8c0cd6adbeac46b34e, ASSIGN}] 2023-08-06 09:53:19,592 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c13a71834623cd8c0cd6adbeac46b34e, ASSIGN 2023-08-06 09:53:19,593 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c13a71834623cd8c0cd6adbeac46b34e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44015,1691315596630; forceNewPlan=false, retain=false 2023-08-06 09:53:19,743 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:53:19,745 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=c13a71834623cd8c0cd6adbeac46b34e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:19,745 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691315599744"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315599744"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315599744"}]},"ts":"1691315599744"} 2023-08-06 09:53:19,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure c13a71834623cd8c0cd6adbeac46b34e, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:19,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:19,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c13a71834623cd8c0cd6adbeac46b34e, NAME => 'testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:53:19,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:19,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:19,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:19,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:19,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:19,913 INFO [StoreOpener-c13a71834623cd8c0cd6adbeac46b34e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:19,915 DEBUG [StoreOpener-c13a71834623cd8c0cd6adbeac46b34e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1 2023-08-06 09:53:19,915 DEBUG [StoreOpener-c13a71834623cd8c0cd6adbeac46b34e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1 2023-08-06 09:53:19,915 INFO [StoreOpener-c13a71834623cd8c0cd6adbeac46b34e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c13a71834623cd8c0cd6adbeac46b34e columnFamilyName cf1 2023-08-06 09:53:19,916 INFO [StoreOpener-c13a71834623cd8c0cd6adbeac46b34e-1] regionserver.HStore(310): Store=c13a71834623cd8c0cd6adbeac46b34e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:19,916 INFO [StoreOpener-c13a71834623cd8c0cd6adbeac46b34e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:19,918 DEBUG [StoreOpener-c13a71834623cd8c0cd6adbeac46b34e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2 2023-08-06 09:53:19,918 DEBUG [StoreOpener-c13a71834623cd8c0cd6adbeac46b34e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2 2023-08-06 09:53:19,919 INFO [StoreOpener-c13a71834623cd8c0cd6adbeac46b34e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c13a71834623cd8c0cd6adbeac46b34e columnFamilyName cf2 2023-08-06 09:53:19,919 INFO [StoreOpener-c13a71834623cd8c0cd6adbeac46b34e-1] regionserver.HStore(310): Store=c13a71834623cd8c0cd6adbeac46b34e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:19,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:19,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:19,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:19,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:53:19,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c13a71834623cd8c0cd6adbeac46b34e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69747942, jitterRate=0.039325326681137085}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:19,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c13a71834623cd8c0cd6adbeac46b34e: 2023-08-06 09:53:19,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e., pid=11, masterSystemTime=1691315599901 2023-08-06 09:53:19,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:19,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:19,936 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=c13a71834623cd8c0cd6adbeac46b34e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:19,936 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691315599936"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315599936"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315599936"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315599936"}]},"ts":"1691315599936"} 2023-08-06 09:53:19,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-06 09:53:19,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure c13a71834623cd8c0cd6adbeac46b34e, server=jenkins-hbase4.apache.org,44015,1691315596630 in 191 msec 2023-08-06 09:53:19,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-06 09:53:19,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c13a71834623cd8c0cd6adbeac46b34e, ASSIGN in 353 msec 2023-08-06 09:53:19,946 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-06 09:53:19,946 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315599946"}]},"ts":"1691315599946"} 2023-08-06 09:53:19,948 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-08-06 09:53:19,951 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-08-06 09:53:19,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 440 msec 2023-08-06 09:53:20,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:20,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,045 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:20,045 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,046 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-06 09:53:20,046 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-06 09:53:20,046 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,046 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,047 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,047 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,048 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,048 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,048 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,048 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:20,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,053 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,053 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,053 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,053 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,053 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,054 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,054 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,053 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,054 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:20,054 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,054 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:20,055 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,055 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:20,055 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:20,056 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:20,056 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:20,056 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:20,056 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. started... 2023-08-06 09:53:20,057 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c13a71834623cd8c0cd6adbeac46b34e 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-06 09:53:20,058 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,058 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,058 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,058 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,058 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:20,058 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:20,134 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/bd90f6e30cb94e228a7a4c7f8d992c69 2023-08-06 09:53:20,201 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/210a6ea95ad741b9b10ffbf51f131209 2023-08-06 09:53:20,212 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/bd90f6e30cb94e228a7a4c7f8d992c69 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69 2023-08-06 09:53:20,223 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:53:20,226 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/210a6ea95ad741b9b10ffbf51f131209 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209 2023-08-06 09:53:20,234 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:53:20,237 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c13a71834623cd8c0cd6adbeac46b34e in 180ms, sequenceid=6, compaction requested=false 2023-08-06 09:53:20,238 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:20,240 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c13a71834623cd8c0cd6adbeac46b34e: 2023-08-06 09:53:20,241 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:20,241 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:20,241 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:20,241 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,241 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:20,241 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:20,244 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,244 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:20,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:20,245 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,245 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:20,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:20,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:20,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:20,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-06 09:53:20,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d3d19f7[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:20,248 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:20,248 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,252 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,253 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,253 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:20,253 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:20,253 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:20,254 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:20,257 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:20,257 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,257 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:20,258 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:20,258 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:20,258 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,258 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,258 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:20,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:20,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:20,258 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,259 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:20,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:20,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:20,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:20,265 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:20,265 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-06 09:53:20,265 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:20,265 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:20,265 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,265 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:20,267 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,268 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,267 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,268 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,268 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,268 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,268 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:20,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:20,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,268 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:20,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,269 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,270 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:20,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:20,273 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,273 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,274 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,291 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,291 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,291 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,291 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:20,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:20,291 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,291 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,291 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,291 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,291 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,293 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,293 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,293 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,293 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,293 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,293 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,293 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,293 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,293 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,295 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:20,295 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:20,395 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:20,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:20,427 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:20,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:20,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,431 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:20,431 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,432 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-06 09:53:20,432 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-06 09:53:20,432 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,432 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,434 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,434 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,434 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,434 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,434 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,435 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,436 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,436 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,436 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,436 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:20,436 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,436 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:20,436 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,436 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,436 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:20,437 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,436 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,437 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:20,437 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,437 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:20,437 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,438 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,438 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,438 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,438 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,439 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,439 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:20,439 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:20,440 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:20,440 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:20,440 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:20,440 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. started... 2023-08-06 09:53:20,441 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c13a71834623cd8c0cd6adbeac46b34e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:20,442 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,443 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,443 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,443 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:20,443 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,443 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:20,462 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/8e18e83139934413bd3d2d93b9d575ba 2023-08-06 09:53:20,497 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/f85840a9de7b4d22871fc9adb238c07b 2023-08-06 09:53:20,506 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/8e18e83139934413bd3d2d93b9d575ba as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba 2023-08-06 09:53:20,515 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:53:20,516 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/f85840a9de7b4d22871fc9adb238c07b as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b 2023-08-06 09:53:20,525 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:53:20,526 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c13a71834623cd8c0cd6adbeac46b34e in 85ms, sequenceid=19, compaction requested=false 2023-08-06 09:53:20,526 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c13a71834623cd8c0cd6adbeac46b34e: 2023-08-06 09:53:20,526 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:20,526 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:20,527 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:20,527 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,527 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:20,527 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:20,529 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,529 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:20,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:20,529 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,529 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:20,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:20,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:20,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:20,532 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-06 09:53:20,532 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@606a92fb[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:20,532 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:20,532 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,535 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,535 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,535 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,535 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,535 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:20,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,535 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:20,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,535 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:20,535 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:20,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,536 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,535 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:20,536 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:20,536 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:20,536 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:20,536 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:20,538 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:20,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:20,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:20,539 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:20,539 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:20,539 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,539 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,539 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:20,540 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:20,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:20,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:20,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:20,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:20,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:20,545 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-06 09:53:20,545 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:20,545 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:20,545 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,545 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:20,547 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,547 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,547 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,547 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,547 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,547 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,548 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:20,548 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:20,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,549 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:20,550 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:20,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,553 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,553 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:20,554 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,564 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,565 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,565 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:20,565 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:20,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,565 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,565 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,566 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,566 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,566 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,566 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,566 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,566 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,566 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:20,566 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:20,565 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,565 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,566 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,666 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:20,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:20,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:20,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:20,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,697 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:20,697 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,697 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-06 09:53:20,698 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-06 09:53:20,698 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,698 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,700 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,700 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,700 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,700 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,700 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,700 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:20,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:20,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,704 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:20,704 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:20,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,705 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,705 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,705 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,705 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,705 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:20,705 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:20,705 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,705 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,706 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,706 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,706 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:20,706 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:20,706 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:20,706 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:20,706 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:20,707 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. started... 2023-08-06 09:53:20,707 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c13a71834623cd8c0cd6adbeac46b34e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:20,708 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,709 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,709 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:20,709 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,709 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,709 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:20,727 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/cd814ee0c3bb462c9a83a8c609a07dfc 2023-08-06 09:53:20,752 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/93a3864147cd40b283f6e5c87b2b3b50 2023-08-06 09:53:20,760 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/cd814ee0c3bb462c9a83a8c609a07dfc as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc 2023-08-06 09:53:20,770 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:53:20,771 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/93a3864147cd40b283f6e5c87b2b3b50 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50 2023-08-06 09:53:20,782 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:53:20,783 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c13a71834623cd8c0cd6adbeac46b34e in 76ms, sequenceid=32, compaction requested=true 2023-08-06 09:53:20,784 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c13a71834623cd8c0cd6adbeac46b34e: 2023-08-06 09:53:20,784 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:20,784 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:20,787 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,787 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:20,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:20,788 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,788 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:20,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:20,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:20,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:20,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-06 09:53:20,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50bc19c6[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:20,790 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:20,790 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,792 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,792 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,792 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,792 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:20,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,792 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,792 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:20,792 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:20,792 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:20,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,792 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:20,792 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:20,792 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:20,792 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:20,793 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:20,793 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:20,796 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:20,796 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,796 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:20,797 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:20,797 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,797 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:20,803 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:20,803 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,803 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:20,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:20,803 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:20,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:20,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:20,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:20,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:20,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:20,808 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-06 09:53:20,809 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:20,809 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:20,809 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,809 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:20,810 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,810 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,811 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,810 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:20,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:20,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,811 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,811 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,811 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:20,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,812 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,813 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:20,814 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:20,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,816 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,817 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,817 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,821 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,821 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,821 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,821 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,821 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,821 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,821 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,821 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,821 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,821 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:20,822 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:20,822 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:20,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:20,822 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:20,822 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:20,823 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:20,823 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:20,823 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:20,823 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:20,823 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,823 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,923 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:20,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:20,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:20,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:20,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,950 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:20,950 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,951 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-06 09:53:20,951 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-06 09:53:20,952 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,952 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,954 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,954 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,954 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,954 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:20,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:20,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,955 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:20,955 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:20,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:20,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:20,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,956 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,956 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,956 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,956 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:20,957 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,957 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:20,957 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,957 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,957 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:20,957 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,957 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,957 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,957 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,958 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,958 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:20,960 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:20,960 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:20,960 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:20,960 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:20,960 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. started... 2023-08-06 09:53:20,960 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c13a71834623cd8c0cd6adbeac46b34e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:20,962 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:20,964 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:20,964 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:20,991 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/ff075565ebcd4a92b42e71da234c12e4 2023-08-06 09:53:21,017 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/e9f6aa952ba0446e9a61a0591605eb3f 2023-08-06 09:53:21,026 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/ff075565ebcd4a92b42e71da234c12e4 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4 2023-08-06 09:53:21,034 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:53:21,035 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/e9f6aa952ba0446e9a61a0591605eb3f as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f 2023-08-06 09:53:21,044 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:53:21,045 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c13a71834623cd8c0cd6adbeac46b34e in 85ms, sequenceid=45, compaction requested=true 2023-08-06 09:53:21,046 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c13a71834623cd8c0cd6adbeac46b34e: 2023-08-06 09:53:21,046 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:21,046 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:21,047 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:21,048 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,048 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:21,048 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:21,052 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,052 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:21,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:21,053 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,053 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:21,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:21,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:21,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:21,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-06 09:53:21,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f29f94c[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:21,056 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:21,057 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,063 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,063 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,063 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,064 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:21,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,064 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:21,064 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:21,064 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:21,064 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:21,064 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:21,064 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,064 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:21,064 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:21,064 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:21,064 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:21,066 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:21,066 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,066 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:21,067 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:21,067 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:21,067 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,067 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,067 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:21,067 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:21,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:21,067 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:21,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:21,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:21,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:21,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:21,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:21,072 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-06 09:53:21,072 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:21,073 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:21,073 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,073 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:21,074 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,074 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,075 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,074 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:21,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:21,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,075 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,075 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,075 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,075 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:21,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,076 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,077 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:21,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:21,079 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,080 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,080 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,089 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,089 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,089 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,089 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,089 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,089 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:21,089 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:21,089 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,089 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,089 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,089 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,089 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,090 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:21,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,090 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,090 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,091 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,091 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:21,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,092 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,092 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,092 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,092 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,092 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,192 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:21,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:21,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:21,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:21,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,219 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:21,219 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:21,220 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-06 09:53:21,220 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-06 09:53:21,221 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,221 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,222 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,222 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,222 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,222 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,223 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,223 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:21,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:21,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,225 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:21,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:21,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:21,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,225 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,225 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:21,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,225 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:21,225 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:21,226 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:21,226 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,226 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:21,227 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:21,227 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:21,227 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:21,227 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:21,227 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,227 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:21,227 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:21,227 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:21,228 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:21,227 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:21,229 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:21,229 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:21,230 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:21,230 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:21,230 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. started... 2023-08-06 09:53:21,230 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c13a71834623cd8c0cd6adbeac46b34e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:21,231 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,232 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,232 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,232 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:21,232 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,232 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:21,253 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/da157ef3eda44a5ab7e9fdd1b628864f 2023-08-06 09:53:21,282 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/9e057f9010a14ddbbaf275bd710729fd 2023-08-06 09:53:21,291 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/da157ef3eda44a5ab7e9fdd1b628864f as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/da157ef3eda44a5ab7e9fdd1b628864f 2023-08-06 09:53:21,298 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/da157ef3eda44a5ab7e9fdd1b628864f, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:53:21,299 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/9e057f9010a14ddbbaf275bd710729fd as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/9e057f9010a14ddbbaf275bd710729fd 2023-08-06 09:53:21,308 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/9e057f9010a14ddbbaf275bd710729fd, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:53:21,309 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c13a71834623cd8c0cd6adbeac46b34e in 79ms, sequenceid=58, compaction requested=true 2023-08-06 09:53:21,309 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c13a71834623cd8c0cd6adbeac46b34e: 2023-08-06 09:53:21,310 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:21,310 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:21,310 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:21,310 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,311 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:21,311 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:21,313 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,313 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:21,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:21,313 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,313 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:21,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:21,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:21,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:21,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-06 09:53:21,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@aeebdff[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:21,316 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:21,316 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,317 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,317 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,317 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:21,318 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,318 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:21,320 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,320 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:21,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:21,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:21,320 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,320 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:21,321 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:21,321 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:21,321 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:21,321 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:21,321 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,322 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:21,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:21,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:21,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:21,326 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-06 09:53:21,326 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:21,326 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:21,326 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,326 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:21,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:21,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:21,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,329 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,330 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,331 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:21,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,332 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:21,333 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,333 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,339 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,339 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,339 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,339 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,339 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,339 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,339 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,339 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,339 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,339 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,339 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:21,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,339 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,340 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:21,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,340 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,341 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,341 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,341 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,341 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,341 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:21,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,341 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:21,442 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:21,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:21,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:21,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:21,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,467 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:21,467 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:21,468 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-06 09:53:21,468 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-06 09:53:21,468 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,469 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,470 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,470 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,470 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,470 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,471 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,471 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:21,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:21,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:21,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:21,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:21,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,472 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:21,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:21,472 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:21,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-06 09:53:21,473 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:21,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,473 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:21,473 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:21,473 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:21,473 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,474 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:21,474 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-06 09:53:21,473 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:21,474 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:21,474 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:21,474 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-06 09:53:21,474 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:21,475 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:21,475 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:21,475 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:21,475 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. started... 2023-08-06 09:53:21,475 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c13a71834623cd8c0cd6adbeac46b34e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:21,477 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,478 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,478 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,478 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:21,478 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,478 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:21,495 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/af0e1487c74648e5a7f1e6d4b0a78955 2023-08-06 09:53:21,516 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/3669d40f10c44c448f184f877d767871 2023-08-06 09:53:21,526 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/af0e1487c74648e5a7f1e6d4b0a78955 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/af0e1487c74648e5a7f1e6d4b0a78955 2023-08-06 09:53:21,533 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/af0e1487c74648e5a7f1e6d4b0a78955, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:53:21,534 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/3669d40f10c44c448f184f877d767871 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/3669d40f10c44c448f184f877d767871 2023-08-06 09:53:21,543 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/3669d40f10c44c448f184f877d767871, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:53:21,544 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c13a71834623cd8c0cd6adbeac46b34e in 69ms, sequenceid=71, compaction requested=true 2023-08-06 09:53:21,545 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c13a71834623cd8c0cd6adbeac46b34e: 2023-08-06 09:53:21,545 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:21,545 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:21,545 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:21,545 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,545 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-06 09:53:21,545 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-06 09:53:21,547 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,547 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:21,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:21,548 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,548 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:21,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:21,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:21,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:21,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-06 09:53:21,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e271b13[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:21,550 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:21,550 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,551 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,552 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,552 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,552 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,552 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:21,552 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:21,552 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:21,552 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:21,552 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:21,552 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:21,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,552 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:21,552 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-06 09:53:21,552 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-06 09:53:21,553 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:21,555 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:21,555 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,555 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:21,555 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:21,556 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-06 09:53:21,556 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,556 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,556 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:21,556 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:21,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:21,556 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-06 09:53:21,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:21,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:21,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:21,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:21,561 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:21,561 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-06 09:53:21,561 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:21,561 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:21,561 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,561 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:21,563 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,563 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,563 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,563 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:21,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,563 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,563 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,564 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:21,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,563 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,565 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:21,566 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:21,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,568 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,568 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:21,568 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:21,572 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-06 09:53:21,572 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:21,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,572 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:21,572 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter because node does not exist (not an error) 2023-08-06 09:53:21,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,572 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:21,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:21,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,573 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:21,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,573 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,573 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:21,574 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:21,573 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:21,574 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:21,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:21,574 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:21,574 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,574 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-06 09:53:21,674 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:21,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:21,696 DEBUG [Listener at localhost/36789] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,703 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:21,704 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:21,704 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:21,704 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:21,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c13a71834623cd8c0cd6adbeac46b34e, UNASSIGN}] 2023-08-06 09:53:21,712 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c13a71834623cd8c0cd6adbeac46b34e, UNASSIGN 2023-08-06 09:53:21,713 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=c13a71834623cd8c0cd6adbeac46b34e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:21,713 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691315601713"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315601713"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315601713"}]},"ts":"1691315601713"} 2023-08-06 09:53:21,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure c13a71834623cd8c0cd6adbeac46b34e, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:21,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:21,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c13a71834623cd8c0cd6adbeac46b34e, disabling compactions & flushes 2023-08-06 09:53:21,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:21,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:21,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. after waiting 0 ms 2023-08-06 09:53:21,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:21,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing c13a71834623cd8c0cd6adbeac46b34e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-06 09:53:21,894 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/ac6b36bd38104eab8bf55341257456a0 2023-08-06 09:53:21,920 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/94406e1de40b44d081270d2a3923faed 2023-08-06 09:53:21,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf1/ac6b36bd38104eab8bf55341257456a0 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ac6b36bd38104eab8bf55341257456a0 2023-08-06 09:53:21,935 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ac6b36bd38104eab8bf55341257456a0, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:53:21,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.tmp/cf2/94406e1de40b44d081270d2a3923faed as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/94406e1de40b44d081270d2a3923faed 2023-08-06 09:53:21,945 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/94406e1de40b44d081270d2a3923faed, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:53:21,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c13a71834623cd8c0cd6adbeac46b34e in 70ms, sequenceid=82, compaction requested=true 2023-08-06 09:53:21,967 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-06 09:53:21,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:21,969 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. 2023-08-06 09:53:21,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c13a71834623cd8c0cd6adbeac46b34e: 2023-08-06 09:53:21,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:21,973 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=c13a71834623cd8c0cd6adbeac46b34e, regionState=CLOSED 2023-08-06 09:53:21,973 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691315601973"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315601973"}]},"ts":"1691315601973"} 2023-08-06 09:53:21,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-06 09:53:21,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure c13a71834623cd8c0cd6adbeac46b34e, server=jenkins-hbase4.apache.org,44015,1691315596630 in 260 msec 2023-08-06 09:53:21,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-06 09:53:21,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c13a71834623cd8c0cd6adbeac46b34e, UNASSIGN in 267 msec 2023-08-06 09:53:22,001 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=c13a71834623cd8c0cd6adbeac46b34e, threads=14 2023-08-06 09:53:22,003 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,003 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/3669d40f10c44c448f184f877d767871 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,003 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,003 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/94406e1de40b44d081270d2a3923faed for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,003 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/9e057f9010a14ddbbaf275bd710729fd for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,004 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,004 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,005 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/af0e1487c74648e5a7f1e6d4b0a78955 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,005 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ac6b36bd38104eab8bf55341257456a0 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,005 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,006 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,006 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,007 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/da157ef3eda44a5ab7e9fdd1b628864f for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,007 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,040 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/94406e1de40b44d081270d2a3923faed, top=true 2023-08-06 09:53:22,040 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/9e057f9010a14ddbbaf275bd710729fd, top=true 2023-08-06 09:53:22,041 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/da157ef3eda44a5ab7e9fdd1b628864f, top=true 2023-08-06 09:53:22,053 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ac6b36bd38104eab8bf55341257456a0, top=true 2023-08-06 09:53:22,057 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/af0e1487c74648e5a7f1e6d4b0a78955, top=true 2023-08-06 09:53:22,058 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/3669d40f10c44c448f184f877d767871, top=true 2023-08-06 09:53:22,085 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.splits/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-9e057f9010a14ddbbaf275bd710729fd for child: 7352fe00040842926aae961d72c3f889, parent: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,085 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/9e057f9010a14ddbbaf275bd710729fd for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,096 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.splits/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-94406e1de40b44d081270d2a3923faed for child: 7352fe00040842926aae961d72c3f889, parent: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,096 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/94406e1de40b44d081270d2a3923faed for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,122 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.splits/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-da157ef3eda44a5ab7e9fdd1b628864f for child: 7352fe00040842926aae961d72c3f889, parent: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,123 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/da157ef3eda44a5ab7e9fdd1b628864f for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,131 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.splits/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-3669d40f10c44c448f184f877d767871 for child: 7352fe00040842926aae961d72c3f889, parent: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,131 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.splits/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-ac6b36bd38104eab8bf55341257456a0 for child: 7352fe00040842926aae961d72c3f889, parent: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,131 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/3669d40f10c44c448f184f877d767871 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,131 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ac6b36bd38104eab8bf55341257456a0 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,181 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/.splits/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-af0e1487c74648e5a7f1e6d4b0a78955 for child: 7352fe00040842926aae961d72c3f889, parent: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,181 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/af0e1487c74648e5a7f1e6d4b0a78955 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,234 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,263 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,277 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,278 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,282 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,290 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,291 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4 for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,298 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f for region: c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:22,298 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region c13a71834623cd8c0cd6adbeac46b34e Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-08-06 09:53:22,358 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:53:22,360 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:53:22,363 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1691315602362"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1691315602362"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1691315602362"}]},"ts":"1691315602362"} 2023-08-06 09:53:22,363 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691315602362"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315602362"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315602362"}]},"ts":"1691315602362"} 2023-08-06 09:53:22,363 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691315602362"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315602362"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315602362"}]},"ts":"1691315602362"} 2023-08-06 09:53:22,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ffe855bd6b405f1f25462543a843a917, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7352fe00040842926aae961d72c3f889, ASSIGN}] 2023-08-06 09:53:22,413 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7352fe00040842926aae961d72c3f889, ASSIGN 2023-08-06 09:53:22,414 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7352fe00040842926aae961d72c3f889, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44015,1691315596630; forceNewPlan=false, retain=false 2023-08-06 09:53:22,415 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ffe855bd6b405f1f25462543a843a917, ASSIGN 2023-08-06 09:53:22,416 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ffe855bd6b405f1f25462543a843a917, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44015,1691315596630; forceNewPlan=false, retain=false 2023-08-06 09:53:22,564 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-06 09:53:22,566 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=ffe855bd6b405f1f25462543a843a917, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:22,566 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691315602565"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315602565"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315602565"}]},"ts":"1691315602565"} 2023-08-06 09:53:22,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure ffe855bd6b405f1f25462543a843a917, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:22,572 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7352fe00040842926aae961d72c3f889, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:22,573 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691315602572"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315602572"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315602572"}]},"ts":"1691315602572"} 2023-08-06 09:53:22,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 7352fe00040842926aae961d72c3f889, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:22,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. 2023-08-06 09:53:22,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ffe855bd6b405f1f25462543a843a917, NAME => 'testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.', STARTKEY => '', ENDKEY => '26'} 2023-08-06 09:53:22,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:22,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:22,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:22,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:22,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:22,728 INFO [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:22,729 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1 2023-08-06 09:53:22,730 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1 2023-08-06 09:53:22,730 INFO [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ffe855bd6b405f1f25462543a843a917 columnFamilyName cf1 2023-08-06 09:53:22,762 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba-bottom 2023-08-06 09:53:22,770 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69-bottom 2023-08-06 09:53:22,776 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc-bottom 2023-08-06 09:53:22,783 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4-bottom 2023-08-06 09:53:22,783 INFO [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(310): Store=ffe855bd6b405f1f25462543a843a917/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:22,784 INFO [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:22,785 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2 2023-08-06 09:53:22,785 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2 2023-08-06 09:53:22,785 INFO [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ffe855bd6b405f1f25462543a843a917 columnFamilyName cf2 2023-08-06 09:53:22,802 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209-bottom 2023-08-06 09:53:22,809 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50-bottom 2023-08-06 09:53:22,816 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f-bottom 2023-08-06 09:53:22,823 DEBUG [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b-bottom 2023-08-06 09:53:22,823 INFO [StoreOpener-ffe855bd6b405f1f25462543a843a917-1] regionserver.HStore(310): Store=ffe855bd6b405f1f25462543a843a917/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:22,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:22,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:22,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:22,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:22,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ffe855bd6b405f1f25462543a843a917; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73035125, jitterRate=0.0883081704378128}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:22,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ffe855bd6b405f1f25462543a843a917: 2023-08-06 09:53:22,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917., pid=17, masterSystemTime=1691315602721 2023-08-06 09:53:22,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:22,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-06 09:53:22,836 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-08-06 09:53:22,836 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-08-06 09:53:22,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. 2023-08-06 09:53:22,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. 2023-08-06 09:53:22,839 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. 2023-08-06 09:53:22,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. 2023-08-06 09:53:22,839 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1912): ffe855bd6b405f1f25462543a843a917/cf1 is initiating minor compaction (all files) 2023-08-06 09:53:22,839 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. 2023-08-06 09:53:22,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7352fe00040842926aae961d72c3f889, NAME => 'testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.', STARTKEY => '26', ENDKEY => ''} 2023-08-06 09:53:22,839 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ffe855bd6b405f1f25462543a843a917/cf1 in testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. 2023-08-06 09:53:22,839 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1912): ffe855bd6b405f1f25462543a843a917/cf2 is initiating minor compaction (all files) 2023-08-06 09:53:22,839 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2259): Starting compaction of ffe855bd6b405f1f25462543a843a917/cf2 in testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. 2023-08-06 09:53:22,839 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=ffe855bd6b405f1f25462543a843a917, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:22,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:22,839 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4-bottom] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/.tmp, totalSize=19.9 K 2023-08-06 09:53:22,840 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f-bottom] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/.tmp, totalSize=19.9 K 2023-08-06 09:53:22,840 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691315602839"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315602839"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315602839"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315602839"}]},"ts":"1691315602839"} 2023-08-06 09:53:22,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 7352fe00040842926aae961d72c3f889 2023-08-06 09:53:22,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:22,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7352fe00040842926aae961d72c3f889 2023-08-06 09:53:22,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7352fe00040842926aae961d72c3f889 2023-08-06 09:53:22,842 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315600019 2023-08-06 09:53:22,842 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315600019 2023-08-06 09:53:22,844 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315600403 2023-08-06 09:53:22,844 INFO [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7352fe00040842926aae961d72c3f889 2023-08-06 09:53:22,844 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315600669 2023-08-06 09:53:22,845 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315600403 2023-08-06 09:53:22,846 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315600926 2023-08-06 09:53:22,847 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315600669 2023-08-06 09:53:22,847 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1 2023-08-06 09:53:22,847 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1 2023-08-06 09:53:22,847 INFO [StoreOpener-7352fe00040842926aae961d72c3f889-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7352fe00040842926aae961d72c3f889 columnFamilyName cf1 2023-08-06 09:53:22,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-08-06 09:53:22,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure ffe855bd6b405f1f25462543a843a917, server=jenkins-hbase4.apache.org,44015,1691315596630 in 276 msec 2023-08-06 09:53:22,849 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315600926 2023-08-06 09:53:22,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ffe855bd6b405f1f25462543a843a917, ASSIGN in 436 msec 2023-08-06 09:53:22,869 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] throttle.PressureAwareThroughputController(145): ffe855bd6b405f1f25462543a843a917#cf1#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:22,869 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] throttle.PressureAwareThroughputController(145): ffe855bd6b405f1f25462543a843a917#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:22,890 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba-top 2023-08-06 09:53:22,898 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69-top 2023-08-06 09:53:22,910 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc-top 2023-08-06 09:53:22,911 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/.tmp/cf1/f4d20525ff374e18ad2d03c7156a981a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/f4d20525ff374e18ad2d03c7156a981a 2023-08-06 09:53:22,915 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/.tmp/cf2/adcc3f501da0473ea4588950588757e4 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/adcc3f501da0473ea4588950588757e4 2023-08-06 09:53:22,919 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4-top 2023-08-06 09:53:22,934 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in ffe855bd6b405f1f25462543a843a917/cf1 of ffe855bd6b405f1f25462543a843a917 into f4d20525ff374e18ad2d03c7156a981a(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:22,934 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in ffe855bd6b405f1f25462543a843a917/cf2 of ffe855bd6b405f1f25462543a843a917 into adcc3f501da0473ea4588950588757e4(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:22,936 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ffe855bd6b405f1f25462543a843a917: 2023-08-06 09:53:22,936 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for ffe855bd6b405f1f25462543a843a917: 2023-08-06 09:53:22,936 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917., storeName=ffe855bd6b405f1f25462543a843a917/cf1, priority=12, startTime=1691315602835; duration=0sec 2023-08-06 09:53:22,936 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917., storeName=ffe855bd6b405f1f25462543a843a917/cf2, priority=12, startTime=1691315602835; duration=0sec 2023-08-06 09:53:22,937 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:22,937 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:22,937 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-ac6b36bd38104eab8bf55341257456a0 2023-08-06 09:53:22,943 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-af0e1487c74648e5a7f1e6d4b0a78955 2023-08-06 09:53:22,953 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-da157ef3eda44a5ab7e9fdd1b628864f 2023-08-06 09:53:22,953 INFO [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(310): Store=7352fe00040842926aae961d72c3f889/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:22,954 INFO [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7352fe00040842926aae961d72c3f889 2023-08-06 09:53:22,955 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2 2023-08-06 09:53:22,955 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2 2023-08-06 09:53:22,956 INFO [StoreOpener-7352fe00040842926aae961d72c3f889-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7352fe00040842926aae961d72c3f889 columnFamilyName cf2 2023-08-06 09:53:22,974 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209-top 2023-08-06 09:53:22,984 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50-top 2023-08-06 09:53:22,994 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f-top 2023-08-06 09:53:23,000 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b-top 2023-08-06 09:53:23,006 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-3669d40f10c44c448f184f877d767871 2023-08-06 09:53:23,012 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-94406e1de40b44d081270d2a3923faed 2023-08-06 09:53:23,019 DEBUG [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-9e057f9010a14ddbbaf275bd710729fd 2023-08-06 09:53:23,020 INFO [StoreOpener-7352fe00040842926aae961d72c3f889-1] regionserver.HStore(310): Store=7352fe00040842926aae961d72c3f889/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:23,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889 2023-08-06 09:53:23,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889 2023-08-06 09:53:23,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:23,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7352fe00040842926aae961d72c3f889 2023-08-06 09:53:23,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7352fe00040842926aae961d72c3f889; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62109065, jitterRate=-0.07450281083583832}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:23,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7352fe00040842926aae961d72c3f889: 2023-08-06 09:53:23,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889., pid=18, masterSystemTime=1691315602721 2023-08-06 09:53:23,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:23,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-06 09:53:23,030 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:23,030 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:23,031 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. 2023-08-06 09:53:23,031 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1912): 7352fe00040842926aae961d72c3f889/cf1 is initiating minor compaction (all files) 2023-08-06 09:53:23,031 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. 2023-08-06 09:53:23,031 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7352fe00040842926aae961d72c3f889/cf1 in testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. 2023-08-06 09:53:23,031 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1912): 7352fe00040842926aae961d72c3f889/cf2 is initiating minor compaction (all files) 2023-08-06 09:53:23,031 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 7352fe00040842926aae961d72c3f889/cf2 in testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. 2023-08-06 09:53:23,032 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-da157ef3eda44a5ab7e9fdd1b628864f, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-af0e1487c74648e5a7f1e6d4b0a78955, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-ac6b36bd38104eab8bf55341257456a0] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/.tmp, totalSize=34.9 K 2023-08-06 09:53:23,032 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-9e057f9010a14ddbbaf275bd710729fd, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-3669d40f10c44c448f184f877d767871, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-94406e1de40b44d081270d2a3923faed] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/.tmp, totalSize=34.9 K 2023-08-06 09:53:23,033 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691315600019 2023-08-06 09:53:23,034 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691315600019 2023-08-06 09:53:23,035 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691315600403 2023-08-06 09:53:23,035 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. 2023-08-06 09:53:23,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. 2023-08-06 09:53:23,036 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691315600403 2023-08-06 09:53:23,036 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7352fe00040842926aae961d72c3f889, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:23,037 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691315603036"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315603036"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315603036"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315603036"}]},"ts":"1691315603036"} 2023-08-06 09:53:23,037 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691315600669 2023-08-06 09:53:23,038 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691315600669 2023-08-06 09:53:23,039 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691315600926 2023-08-06 09:53:23,040 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691315600926 2023-08-06 09:53:23,041 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-da157ef3eda44a5ab7e9fdd1b628864f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315601195 2023-08-06 09:53:23,041 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-af0e1487c74648e5a7f1e6d4b0a78955, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315601444 2023-08-06 09:53:23,042 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-9e057f9010a14ddbbaf275bd710729fd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315601195 2023-08-06 09:53:23,043 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-ac6b36bd38104eab8bf55341257456a0, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315601676 2023-08-06 09:53:23,044 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-3669d40f10c44c448f184f877d767871, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315601444 2023-08-06 09:53:23,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-08-06 09:53:23,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 7352fe00040842926aae961d72c3f889, server=jenkins-hbase4.apache.org,44015,1691315596630 in 468 msec 2023-08-06 09:53:23,046 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-94406e1de40b44d081270d2a3923faed, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315601676 2023-08-06 09:53:23,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-08-06 09:53:23,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7352fe00040842926aae961d72c3f889, ASSIGN in 633 msec 2023-08-06 09:53:23,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 in 1.3530 sec 2023-08-06 09:53:23,077 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7352fe00040842926aae961d72c3f889#cf1#compaction#16 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:23,081 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] throttle.PressureAwareThroughputController(145): 7352fe00040842926aae961d72c3f889#cf2#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:23,103 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/.tmp/cf1/ffbd8b96f6ab4c9fa2c649d9c2eca526 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/ffbd8b96f6ab4c9fa2c649d9c2eca526 2023-08-06 09:53:23,107 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/.tmp/cf2/2532a97913ba4e45a004808f24a759e1 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/2532a97913ba4e45a004808f24a759e1 2023-08-06 09:53:23,112 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7352fe00040842926aae961d72c3f889/cf1 of 7352fe00040842926aae961d72c3f889 into ffbd8b96f6ab4c9fa2c649d9c2eca526(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:23,113 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7352fe00040842926aae961d72c3f889: 2023-08-06 09:53:23,113 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889., storeName=7352fe00040842926aae961d72c3f889/cf1, priority=9, startTime=1691315603029; duration=0sec 2023-08-06 09:53:23,113 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:23,116 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7352fe00040842926aae961d72c3f889/cf2 of 7352fe00040842926aae961d72c3f889 into 2532a97913ba4e45a004808f24a759e1(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:23,118 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7352fe00040842926aae961d72c3f889: 2023-08-06 09:53:23,119 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889., storeName=7352fe00040842926aae961d72c3f889/cf2, priority=9, startTime=1691315603030; duration=0sec 2023-08-06 09:53:23,119 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:23,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44015] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:54744 deadline: 1691315663205, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e. is not online on jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:23,339 DEBUG [Listener at localhost/36789] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-08-06 09:53:23,349 INFO [Listener at localhost/36789] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-06 09:53:23,351 DEBUG [Listener at localhost/36789] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-08-06 09:53:23,385 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:23,399 INFO [Listener at localhost/36789] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-08-06 09:53:23,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-08-06 09:53:23,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:23,419 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315603419"}]},"ts":"1691315603419"} 2023-08-06 09:53:23,421 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-08-06 09:53:23,424 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-08-06 09:53:23,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-08-06 09:53:23,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ffe855bd6b405f1f25462543a843a917, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7352fe00040842926aae961d72c3f889, UNASSIGN}] 2023-08-06 09:53:23,428 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7352fe00040842926aae961d72c3f889, UNASSIGN 2023-08-06 09:53:23,429 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7352fe00040842926aae961d72c3f889, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:23,429 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691315603429"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315603429"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315603429"}]},"ts":"1691315603429"} 2023-08-06 09:53:23,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 7352fe00040842926aae961d72c3f889, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:23,436 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ffe855bd6b405f1f25462543a843a917, UNASSIGN 2023-08-06 09:53:23,437 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=ffe855bd6b405f1f25462543a843a917, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:23,437 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691315603437"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315603437"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315603437"}]},"ts":"1691315603437"} 2023-08-06 09:53:23,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure ffe855bd6b405f1f25462543a843a917, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:23,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-08-06 09:53:23,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:23,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ffe855bd6b405f1f25462543a843a917, disabling compactions & flushes 2023-08-06 09:53:23,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. 2023-08-06 09:53:23,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. 2023-08-06 09:53:23,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. after waiting 0 ms 2023-08-06 09:53:23,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. 2023-08-06 09:53:23,590 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4-bottom] to archive 2023-08-06 09:53:23,592 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:23,601 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,603 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,606 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,608 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,635 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f-bottom] to archive 2023-08-06 09:53:23,636 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:23,638 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,641 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,643 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,644 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,650 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-06 09:53:23,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:23,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917. 2023-08-06 09:53:23,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ffe855bd6b405f1f25462543a843a917: 2023-08-06 09:53:23,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:23,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 7352fe00040842926aae961d72c3f889 2023-08-06 09:53:23,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7352fe00040842926aae961d72c3f889, disabling compactions & flushes 2023-08-06 09:53:23,655 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. 2023-08-06 09:53:23,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. 2023-08-06 09:53:23,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. after waiting 0 ms 2023-08-06 09:53:23,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. 2023-08-06 09:53:23,655 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=ffe855bd6b405f1f25462543a843a917, regionState=CLOSED 2023-08-06 09:53:23,655 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691315603655"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315603655"}]},"ts":"1691315603655"} 2023-08-06 09:53:23,658 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-da157ef3eda44a5ab7e9fdd1b628864f, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-af0e1487c74648e5a7f1e6d4b0a78955, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-ac6b36bd38104eab8bf55341257456a0] to archive 2023-08-06 09:53:23,659 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:23,662 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/bd90f6e30cb94e228a7a4c7f8d992c69.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-08-06 09:53:23,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure ffe855bd6b405f1f25462543a843a917, server=jenkins-hbase4.apache.org,44015,1691315596630 in 219 msec 2023-08-06 09:53:23,665 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/8e18e83139934413bd3d2d93b9d575ba.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ffe855bd6b405f1f25462543a843a917, UNASSIGN in 236 msec 2023-08-06 09:53:23,667 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/cd814ee0c3bb462c9a83a8c609a07dfc.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,669 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/ff075565ebcd4a92b42e71da234c12e4.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,671 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-da157ef3eda44a5ab7e9fdd1b628864f to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-da157ef3eda44a5ab7e9fdd1b628864f 2023-08-06 09:53:23,672 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-af0e1487c74648e5a7f1e6d4b0a78955 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-af0e1487c74648e5a7f1e6d4b0a78955 2023-08-06 09:53:23,674 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-ac6b36bd38104eab8bf55341257456a0 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-ac6b36bd38104eab8bf55341257456a0 2023-08-06 09:53:23,676 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-9e057f9010a14ddbbaf275bd710729fd, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-3669d40f10c44c448f184f877d767871, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-94406e1de40b44d081270d2a3923faed] to archive 2023-08-06 09:53:23,677 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:23,680 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/210a6ea95ad741b9b10ffbf51f131209.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,682 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/f85840a9de7b4d22871fc9adb238c07b.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,684 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/93a3864147cd40b283f6e5c87b2b3b50.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,685 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/e9f6aa952ba0446e9a61a0591605eb3f.c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,687 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-9e057f9010a14ddbbaf275bd710729fd to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-9e057f9010a14ddbbaf275bd710729fd 2023-08-06 09:53:23,689 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-3669d40f10c44c448f184f877d767871 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-3669d40f10c44c448f184f877d767871 2023-08-06 09:53:23,690 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-94406e1de40b44d081270d2a3923faed to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/testSplitTableRegionUnevenDaughter=c13a71834623cd8c0cd6adbeac46b34e-94406e1de40b44d081270d2a3923faed 2023-08-06 09:53:23,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-06 09:53:23,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:23,700 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889. 2023-08-06 09:53:23,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7352fe00040842926aae961d72c3f889: 2023-08-06 09:53:23,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 7352fe00040842926aae961d72c3f889 2023-08-06 09:53:23,704 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7352fe00040842926aae961d72c3f889, regionState=CLOSED 2023-08-06 09:53:23,704 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1691315603704"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315603704"}]},"ts":"1691315603704"} 2023-08-06 09:53:23,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-08-06 09:53:23,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 7352fe00040842926aae961d72c3f889, server=jenkins-hbase4.apache.org,44015,1691315596630 in 276 msec 2023-08-06 09:53:23,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-08-06 09:53:23,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7352fe00040842926aae961d72c3f889, UNASSIGN in 284 msec 2023-08-06 09:53:23,715 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315603714"}]},"ts":"1691315603714"} 2023-08-06 09:53:23,716 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-08-06 09:53:23,718 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-08-06 09:53:23,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 312 msec 2023-08-06 09:53:23,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-08-06 09:53:23,734 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-08-06 09:53:23,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-08-06 09:53:23,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:23,748 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:23,749 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:23,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-08-06 09:53:23,758 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,758 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:23,758 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889 2023-08-06 09:53:23,762 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/recovered.edits] 2023-08-06 09:53:23,762 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/recovered.edits] 2023-08-06 09:53:23,763 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/recovered.edits] 2023-08-06 09:53:23,768 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/f4d20525ff374e18ad2d03c7156a981a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf1/f4d20525ff374e18ad2d03c7156a981a 2023-08-06 09:53:23,769 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/ffbd8b96f6ab4c9fa2c649d9c2eca526 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf1/ffbd8b96f6ab4c9fa2c649d9c2eca526 2023-08-06 09:53:23,770 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/8e18e83139934413bd3d2d93b9d575ba 2023-08-06 09:53:23,772 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ac6b36bd38104eab8bf55341257456a0 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ac6b36bd38104eab8bf55341257456a0 2023-08-06 09:53:23,773 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/adcc3f501da0473ea4588950588757e4 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/cf2/adcc3f501da0473ea4588950588757e4 2023-08-06 09:53:23,777 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/af0e1487c74648e5a7f1e6d4b0a78955 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/af0e1487c74648e5a7f1e6d4b0a78955 2023-08-06 09:53:23,777 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/2532a97913ba4e45a004808f24a759e1 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/cf2/2532a97913ba4e45a004808f24a759e1 2023-08-06 09:53:23,778 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/recovered.edits/90.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917/recovered.edits/90.seqid 2023-08-06 09:53:23,780 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/bd90f6e30cb94e228a7a4c7f8d992c69 2023-08-06 09:53:23,781 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/ffe855bd6b405f1f25462543a843a917 2023-08-06 09:53:23,785 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/cd814ee0c3bb462c9a83a8c609a07dfc 2023-08-06 09:53:23,786 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/recovered.edits/90.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889/recovered.edits/90.seqid 2023-08-06 09:53:23,787 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/7352fe00040842926aae961d72c3f889 2023-08-06 09:53:23,788 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/da157ef3eda44a5ab7e9fdd1b628864f to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/da157ef3eda44a5ab7e9fdd1b628864f 2023-08-06 09:53:23,789 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf1/ff075565ebcd4a92b42e71da234c12e4 2023-08-06 09:53:23,792 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/210a6ea95ad741b9b10ffbf51f131209 2023-08-06 09:53:23,794 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/3669d40f10c44c448f184f877d767871 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/3669d40f10c44c448f184f877d767871 2023-08-06 09:53:23,795 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/93a3864147cd40b283f6e5c87b2b3b50 2023-08-06 09:53:23,797 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/94406e1de40b44d081270d2a3923faed to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/94406e1de40b44d081270d2a3923faed 2023-08-06 09:53:23,799 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/9e057f9010a14ddbbaf275bd710729fd to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/9e057f9010a14ddbbaf275bd710729fd 2023-08-06 09:53:23,800 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/e9f6aa952ba0446e9a61a0591605eb3f 2023-08-06 09:53:23,802 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/cf2/f85840a9de7b4d22871fc9adb238c07b 2023-08-06 09:53:23,805 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/recovered.edits/85.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e/recovered.edits/85.seqid 2023-08-06 09:53:23,805 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionUnevenDaughter/c13a71834623cd8c0cd6adbeac46b34e 2023-08-06 09:53:23,805 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-08-06 09:53:23,808 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:23,829 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-08-06 09:53:23,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-08-06 09:53:23,871 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-08-06 09:53:23,873 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:23,873 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-08-06 09:53:23,873 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315603873"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:23,873 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315603873"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:23,873 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315603873"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:23,876 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-06 09:53:23,876 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c13a71834623cd8c0cd6adbeac46b34e, NAME => 'testSplitTableRegionUnevenDaughter,,1691315599511.c13a71834623cd8c0cd6adbeac46b34e.', STARTKEY => '', ENDKEY => ''}, {ENCODED => ffe855bd6b405f1f25462543a843a917, NAME => 'testSplitTableRegionUnevenDaughter,,1691315601696.ffe855bd6b405f1f25462543a843a917.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 7352fe00040842926aae961d72c3f889, NAME => 'testSplitTableRegionUnevenDaughter,26,1691315601696.7352fe00040842926aae961d72c3f889.', STARTKEY => '26', ENDKEY => ''}] 2023-08-06 09:53:23,877 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-08-06 09:53:23,877 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691315603877"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:23,879 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-08-06 09:53:23,881 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:23,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 141 msec 2023-08-06 09:53:24,001 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-06 09:53:24,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-08-06 09:53:24,054 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-08-06 09:53:24,075 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-06 09:53:24,075 INFO [Listener at localhost/36789] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=392 (was 361) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:44526 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:46956 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:44788 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:44538 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x709851dd-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:47156 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:37364 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:37344 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:44784 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x709851dd-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:37380 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:37402 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:44810 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:54846 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:44552 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:39750 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:37352 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x709851dd-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:44015-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:44800 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=824 (was 700) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=291 (was 264) - SystemLoadAverage LEAK? -, ProcessCount=175 (was 175), AvailableMemoryMB=2457 (was 2542) 2023-08-06 09:53:24,076 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-06 09:53:24,090 INFO [Listener at localhost/36789] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=392, OpenFileDescriptor=824, MaxFileDescriptor=60000, SystemLoadAverage=291, ProcessCount=175, AvailableMemoryMB=2456 2023-08-06 09:53:24,090 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:24,096 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-06 09:53:24,097 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-06 09:53:24,099 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:24,101 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-08-06 09:53:24,102 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-06 09:53:24,104 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:24,105 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558 empty. 2023-08-06 09:53:24,105 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:24,105 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-08-06 09:53:24,117 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-08-06 09:53:24,119 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => 733289ef46def4ea49c9f7f8372be558, NAME => 'testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp 2023-08-06 09:53:24,128 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:24,128 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing 733289ef46def4ea49c9f7f8372be558, disabling compactions & flushes 2023-08-06 09:53:24,128 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:24,128 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:24,128 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. after waiting 0 ms 2023-08-06 09:53:24,128 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:24,129 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:24,129 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for 733289ef46def4ea49c9f7f8372be558: 2023-08-06 09:53:24,131 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-08-06 09:53:24,132 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315604132"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315604132"}]},"ts":"1691315604132"} 2023-08-06 09:53:24,134 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-06 09:53:24,135 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-06 09:53:24,135 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315604135"}]},"ts":"1691315604135"} 2023-08-06 09:53:24,137 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-08-06 09:53:24,140 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:53:24,141 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:53:24,141 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:53:24,141 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:53:24,141 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:53:24,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, ASSIGN}] 2023-08-06 09:53:24,144 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, ASSIGN 2023-08-06 09:53:24,145 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44015,1691315596630; forceNewPlan=false, retain=false 2023-08-06 09:53:24,295 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:53:24,296 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=733289ef46def4ea49c9f7f8372be558, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,296 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315604296"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315604296"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315604296"}]},"ts":"1691315604296"} 2023-08-06 09:53:24,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:24,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:24,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 733289ef46def4ea49c9f7f8372be558, NAME => 'testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:53:24,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:24,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:24,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:24,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:24,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:24,457 INFO [StoreOpener-733289ef46def4ea49c9f7f8372be558-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:24,459 DEBUG [StoreOpener-733289ef46def4ea49c9f7f8372be558-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1 2023-08-06 09:53:24,459 DEBUG [StoreOpener-733289ef46def4ea49c9f7f8372be558-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1 2023-08-06 09:53:24,459 INFO [StoreOpener-733289ef46def4ea49c9f7f8372be558-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 733289ef46def4ea49c9f7f8372be558 columnFamilyName cf1 2023-08-06 09:53:24,460 INFO [StoreOpener-733289ef46def4ea49c9f7f8372be558-1] regionserver.HStore(310): Store=733289ef46def4ea49c9f7f8372be558/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:24,460 INFO [StoreOpener-733289ef46def4ea49c9f7f8372be558-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:24,462 DEBUG [StoreOpener-733289ef46def4ea49c9f7f8372be558-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2 2023-08-06 09:53:24,462 DEBUG [StoreOpener-733289ef46def4ea49c9f7f8372be558-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2 2023-08-06 09:53:24,463 INFO [StoreOpener-733289ef46def4ea49c9f7f8372be558-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 733289ef46def4ea49c9f7f8372be558 columnFamilyName cf2 2023-08-06 09:53:24,464 INFO [StoreOpener-733289ef46def4ea49c9f7f8372be558-1] regionserver.HStore(310): Store=733289ef46def4ea49c9f7f8372be558/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:24,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:24,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:24,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:24,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:24,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:53:24,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 733289ef46def4ea49c9f7f8372be558; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63639000, jitterRate=-0.051705002784729004}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:24,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 733289ef46def4ea49c9f7f8372be558: 2023-08-06 09:53:24,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558., pid=27, masterSystemTime=1691315604451 2023-08-06 09:53:24,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:24,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:24,476 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=733289ef46def4ea49c9f7f8372be558, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,476 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315604476"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315604476"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315604476"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315604476"}]},"ts":"1691315604476"} 2023-08-06 09:53:24,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-08-06 09:53:24,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 in 179 msec 2023-08-06 09:53:24,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-08-06 09:53:24,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, ASSIGN in 339 msec 2023-08-06 09:53:24,484 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-06 09:53:24,485 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315604485"}]},"ts":"1691315604485"} 2023-08-06 09:53:24,486 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-08-06 09:53:24,488 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-08-06 09:53:24,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 392 msec 2023-08-06 09:53:24,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:24,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-06 09:53:24,614 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-06 09:53:24,614 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:24,615 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-06 09:53:24,615 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-06 09:53:24,616 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,616 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,618 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,618 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,618 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,618 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,618 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,618 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:24,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:24,620 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:24,620 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:24,620 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:24,620 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:24,621 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:24,621 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:24,620 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:24,622 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:24,623 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:24,622 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:24,623 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:24,624 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. started... 2023-08-06 09:53:24,624 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:24,624 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:24,624 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:24,624 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:24,624 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:24,624 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:24,626 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,626 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,626 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,626 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:24,627 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,627 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:24,628 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 733289ef46def4ea49c9f7f8372be558 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-06 09:53:24,641 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/83b035add6304b7c9f4d2a5323538112 2023-08-06 09:53:24,662 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/c74948662c434ac3a7aa8808e41d4f00 2023-08-06 09:53:24,668 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/83b035add6304b7c9f4d2a5323538112 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112 2023-08-06 09:53:24,674 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:53:24,675 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/c74948662c434ac3a7aa8808e41d4f00 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00 2023-08-06 09:53:24,681 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:53:24,682 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 733289ef46def4ea49c9f7f8372be558 in 54ms, sequenceid=6, compaction requested=false 2023-08-06 09:53:24,682 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-08-06 09:53:24,683 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 733289ef46def4ea49c9f7f8372be558: 2023-08-06 09:53:24,683 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:24,683 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:24,683 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:24,683 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,683 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:24,683 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:24,685 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,685 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:24,686 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,686 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:24,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-06 09:53:24,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ee55738[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:24,688 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-06 09:53:24,688 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,689 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,690 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,690 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,690 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,690 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:24,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,690 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:24,690 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:24,690 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:24,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,690 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:24,690 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:24,690 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:24,690 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:24,690 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:24,690 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:24,693 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:24,693 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:24,693 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:24,694 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:24,694 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,694 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:24,694 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:24,694 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:24,695 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:24,695 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:24,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:24,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:24,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:24,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-06 09:53:24,704 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-06 09:53:24,704 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:24,704 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:24,704 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-06 09:53:24,704 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:24,706 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,706 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,706 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,706 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,706 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,706 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:24,707 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,707 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:24,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,708 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:24,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,709 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,711 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:24,711 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,711 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,714 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,714 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,714 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,714 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,715 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,715 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,714 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,715 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,715 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:24,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,715 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-06 09:53:24,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:24,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,715 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,717 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,717 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,717 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,717 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,717 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,717 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:24,717 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:24,717 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,717 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,817 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:24,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:24,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:24,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:24,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-06 09:53:24,844 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-06 09:53:24,844 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:24,844 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-06 09:53:24,844 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-06 09:53:24,845 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,845 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,847 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,847 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,847 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,847 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,847 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,848 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:24,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,849 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:24,849 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:24,849 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,849 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:24,849 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:24,849 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:24,849 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:24,850 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:24,850 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:24,850 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:24,850 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:24,850 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:24,851 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:24,851 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:24,851 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:24,851 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:24,851 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:24,851 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:24,851 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:24,851 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:24,851 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:24,851 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:24,852 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:24,852 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:24,852 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:24,852 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:24,852 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. started... 2023-08-06 09:53:24,853 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 733289ef46def4ea49c9f7f8372be558 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:24,853 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,854 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,854 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:24,854 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,854 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,854 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:24,872 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/aabf6e71d5fd418890cf8c1989bf21c9 2023-08-06 09:53:24,891 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/e87856bd9e6f438f99e5f713c8cb0e8c 2023-08-06 09:53:24,897 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/aabf6e71d5fd418890cf8c1989bf21c9 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9 2023-08-06 09:53:24,902 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:53:24,903 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/e87856bd9e6f438f99e5f713c8cb0e8c as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c 2023-08-06 09:53:24,908 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:53:24,909 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 733289ef46def4ea49c9f7f8372be558 in 56ms, sequenceid=19, compaction requested=false 2023-08-06 09:53:24,909 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 733289ef46def4ea49c9f7f8372be558: 2023-08-06 09:53:24,909 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:24,909 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:24,910 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:24,910 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,910 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:24,910 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:24,911 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,911 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:24,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:24,912 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,912 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:24,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:24,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:24,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:24,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-06 09:53:24,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ef62c31[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:24,914 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-06 09:53:24,914 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,916 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,916 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,916 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,916 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,917 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:24,917 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:24,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,917 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:24,917 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:24,917 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:24,917 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:24,917 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:24,917 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:24,917 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:24,917 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:24,921 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:24,921 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:24,921 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:24,921 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:24,921 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:24,922 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:24,923 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:24,923 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:24,923 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,923 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:24,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:24,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:24,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:24,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:24,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:24,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-06 09:53:24,929 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-06 09:53:24,929 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:24,929 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:24,929 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-06 09:53:24,929 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:24,932 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,932 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,932 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,932 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,932 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,932 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,932 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,932 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:24,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:24,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,933 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:24,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:24,934 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:24,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,936 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:24,937 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:24,937 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:24,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:24,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-06 09:53:24,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:24,941 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:24,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:24,941 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:24,942 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,942 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:24,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:24,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:24,942 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:24,942 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:24,943 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:24,943 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:24,943 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,043 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:25,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:25,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:25,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:25,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-06 09:53:25,064 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-06 09:53:25,064 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,065 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-06 09:53:25,065 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-06 09:53:25,065 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,066 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,067 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,067 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,067 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,067 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,067 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,068 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:25,068 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,068 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,068 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,068 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,069 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,069 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,069 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,069 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,069 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,069 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,069 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,070 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,070 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,070 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,070 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,070 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,070 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,070 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,070 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,071 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,072 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,072 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,070 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,072 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:25,072 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,072 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:25,072 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,072 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. started... 2023-08-06 09:53:25,072 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:25,072 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 733289ef46def4ea49c9f7f8372be558 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:25,073 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,073 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,074 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,074 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,074 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,074 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,089 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/d66b09b9b2fa4b8b8451a57f452f0fda 2023-08-06 09:53:25,110 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/2d46af24b3444a328be064a53e415ff6 2023-08-06 09:53:25,116 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/d66b09b9b2fa4b8b8451a57f452f0fda as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda 2023-08-06 09:53:25,122 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:53:25,123 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/2d46af24b3444a328be064a53e415ff6 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6 2023-08-06 09:53:25,129 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:53:25,130 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 733289ef46def4ea49c9f7f8372be558 in 58ms, sequenceid=32, compaction requested=true 2023-08-06 09:53:25,130 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 733289ef46def4ea49c9f7f8372be558: 2023-08-06 09:53:25,131 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:25,131 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:25,131 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:25,131 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,131 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,131 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,133 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,133 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,134 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,134 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-06 09:53:25,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1087755[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:25,136 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-06 09:53:25,136 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,138 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,138 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,138 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,138 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,138 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,138 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:25,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,138 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,138 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,138 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:25,138 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,138 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,138 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:25,139 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,139 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:25,140 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,140 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,140 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,141 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,141 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,141 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,141 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,141 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,141 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,141 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:25,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-06 09:53:25,144 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-06 09:53:25,144 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:25,144 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:25,145 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-06 09:53:25,145 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:25,147 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,147 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,147 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,147 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,147 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,147 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,147 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,148 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,149 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,149 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,152 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,152 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,153 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,156 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,156 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,156 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,156 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,156 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,156 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,156 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-06 09:53:25,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,156 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,156 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,156 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,156 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,157 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:25,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,157 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,157 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,157 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,157 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,157 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,157 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,158 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:25,158 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:25,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,258 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:25,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:25,278 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:25,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:25,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-06 09:53:25,280 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-06 09:53:25,280 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,281 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-06 09:53:25,281 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-06 09:53:25,282 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,282 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,283 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,283 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,283 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,283 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,284 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,284 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:25,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,284 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,285 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,285 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,285 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,286 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,286 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,286 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,286 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,287 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,287 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,286 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,287 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,287 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,289 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,289 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,289 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,289 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,289 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:25,289 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:25,290 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:25,290 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. started... 2023-08-06 09:53:25,290 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 733289ef46def4ea49c9f7f8372be558 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:25,295 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,296 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,296 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,296 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,297 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,297 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,304 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/23e5f7c218364a329662875464135793 2023-08-06 09:53:25,322 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6 2023-08-06 09:53:25,328 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/23e5f7c218364a329662875464135793 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793 2023-08-06 09:53:25,334 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:53:25,335 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6 2023-08-06 09:53:25,340 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:53:25,341 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 733289ef46def4ea49c9f7f8372be558 in 51ms, sequenceid=45, compaction requested=true 2023-08-06 09:53:25,341 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 733289ef46def4ea49c9f7f8372be558: 2023-08-06 09:53:25,341 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:25,341 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:25,341 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:25,341 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,341 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,341 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,343 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,343 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,343 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,343 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-06 09:53:25,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7710be4[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:25,346 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-06 09:53:25,346 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,347 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,347 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,347 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,347 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,347 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,347 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:25,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,347 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,347 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,347 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,347 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,347 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,347 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:25,348 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:25,347 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:25,349 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,349 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,350 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,350 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,350 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,350 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,351 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,351 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,351 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,351 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:25,354 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-06 09:53:25,354 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-06 09:53:25,354 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:25,354 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:25,354 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-06 09:53:25,354 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:25,356 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,356 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,356 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,356 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,356 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,356 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,356 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,356 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,357 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,358 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,359 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,360 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,360 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,364 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,364 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,364 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,364 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,364 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,364 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,364 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,364 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,364 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,364 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-06 09:53:25,365 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:25,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,365 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,365 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,365 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,365 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:25,365 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:25,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,365 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,366 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,366 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,366 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,366 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,465 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:25,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:25,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:25,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:25,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-06 09:53:25,489 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-06 09:53:25,490 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,490 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-06 09:53:25,490 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-06 09:53:25,490 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,490 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,492 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,492 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,492 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,492 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,492 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,492 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:25,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,494 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,494 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,494 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,494 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,494 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,494 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,495 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,495 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,494 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,495 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,495 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,495 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,495 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,495 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,495 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,495 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,495 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:25,495 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:25,495 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:25,495 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. started... 2023-08-06 09:53:25,495 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 733289ef46def4ea49c9f7f8372be558 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:25,496 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,497 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,497 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,497 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,498 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,498 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,510 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/a4206353878444f987d0444fc34e0c7d 2023-08-06 09:53:25,516 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-06 09:53:25,516 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-06 09:53:25,517 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:25,517 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-08-06 09:53:25,529 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/45796fef012846449dbfbfb06142b1bc 2023-08-06 09:53:25,534 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/a4206353878444f987d0444fc34e0c7d as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d 2023-08-06 09:53:25,539 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:53:25,540 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/45796fef012846449dbfbfb06142b1bc as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc 2023-08-06 09:53:25,547 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:53:25,548 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 733289ef46def4ea49c9f7f8372be558 in 53ms, sequenceid=58, compaction requested=true 2023-08-06 09:53:25,549 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 733289ef46def4ea49c9f7f8372be558: 2023-08-06 09:53:25,549 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:25,549 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:25,549 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:25,549 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,549 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,549 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,551 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,551 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,552 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,552 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-06 09:53:25,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2309d0a5[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:25,554 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-06 09:53:25,554 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,556 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,556 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,556 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,556 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,556 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,556 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,556 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,556 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,556 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:25,556 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:25,556 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:25,556 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,557 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,557 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:25,558 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,558 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,558 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,559 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,559 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,559 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,559 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,559 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,559 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,559 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:25,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-06 09:53:25,563 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-06 09:53:25,563 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:25,563 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:25,563 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-06 09:53:25,563 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:25,565 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,565 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,565 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,565 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,565 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,566 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,566 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,566 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,567 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,567 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,569 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,569 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,570 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,577 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,577 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,577 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,577 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,577 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,577 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,577 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,577 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,577 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,577 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,578 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,578 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,578 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,578 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,578 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-06 09:53:25,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:25,577 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,578 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,577 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,578 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:25,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,579 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:25,679 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:25,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:25,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:25,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:25,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-06 09:53:25,703 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-06 09:53:25,704 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,704 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-06 09:53:25,704 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-06 09:53:25,704 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,704 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,705 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,705 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,705 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,705 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,706 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,706 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:25,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,707 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-06 09:53:25,708 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,708 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,708 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,708 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-06 09:53:25,708 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,708 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:25,708 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:25,708 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,708 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. started... 2023-08-06 09:53:25,709 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 733289ef46def4ea49c9f7f8372be558 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:25,712 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,717 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,717 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,717 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,717 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,717 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,722 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/2dce879a7489455cad671040392443a5 2023-08-06 09:53:25,738 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/86122f661a594f42a5ef8d7ae8b76b5e 2023-08-06 09:53:25,745 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/2dce879a7489455cad671040392443a5 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5 2023-08-06 09:53:25,751 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:53:25,752 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/86122f661a594f42a5ef8d7ae8b76b5e as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e 2023-08-06 09:53:25,757 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:53:25,758 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 733289ef46def4ea49c9f7f8372be558 in 49ms, sequenceid=71, compaction requested=true 2023-08-06 09:53:25,758 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 733289ef46def4ea49c9f7f8372be558: 2023-08-06 09:53:25,758 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:25,758 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:25,758 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:25,758 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,758 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-06 09:53:25,758 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-06 09:53:25,760 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,760 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,760 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,760 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-06 09:53:25,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6cd1f751[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:25,762 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-06 09:53:25,762 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,763 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,763 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,763 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,763 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,763 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,763 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,763 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,763 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:25,763 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,764 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:25,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,763 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:25,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,764 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-06 09:53:25,764 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-06 09:53:25,764 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,766 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-06 09:53:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,767 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-06 09:53:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:25,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-06 09:53:25,769 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-06 09:53:25,769 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:25,769 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:25,769 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-06 09:53:25,769 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:25,771 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,771 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,771 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,771 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,771 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,771 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,771 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,772 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:25,772 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,773 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,774 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:25,775 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-06 09:53:25,775 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,778 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,778 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,778 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,778 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,778 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,778 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:25,778 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,778 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:25,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,778 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:25,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,778 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:25,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-06 09:53:25,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,778 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:25,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:25,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:25,779 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-06 09:53:25,779 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:25,779 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,779 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:25,779 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:25,779 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-06 09:53:25,779 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-06 09:53:25,779 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:25,779 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:25,879 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:25,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:26,148 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-08-06 09:53:26,149 DEBUG [Listener at localhost/36789] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:26,151 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,152 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,152 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,152 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,153 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:26,153 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:26,153 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:26,401 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,403 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:26,403 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:26,404 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:26,406 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-08-06 09:53:26,407 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:26,407 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:26,407 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:26,407 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:26,408 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:26,411 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-08-06 09:53:26,413 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:26,414 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:26,414 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:26,415 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:26,415 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:26,416 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:26,416 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:26,417 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:26,417 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:26,418 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,418 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:26,418 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,419 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,419 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,419 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-08-06 09:53:26,420 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:26,421 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:26,422 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315636422 2023-08-06 09:53:26,423 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:26,423 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,428 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:26,428 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=733289ef46def4ea49c9f7f8372be558, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:26,428 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:26,428 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315666428 2023-08-06 09:53:26,429 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315726429 2023-08-06 09:53:26,429 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-06 09:53:26,429 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:26,431 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:26,431 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:26,431 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:26,679 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,679 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:26,679 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:26,679 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:26,679 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:26,680 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:26,680 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:26,680 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:26,680 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:26,681 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:26,682 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-06 09:53:26,683 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:26,683 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:26,683 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:26,683 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:26,684 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:26,684 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:26,684 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:26,685 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:26,685 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:26,685 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,685 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:26,685 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,686 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,686 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,686 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-06 09:53:26,686 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:26,687 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:26,687 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315636687 2023-08-06 09:53:26,687 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:26,689 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,692 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:26,692 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=733289ef46def4ea49c9f7f8372be558, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:26,692 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:26,692 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315666692 2023-08-06 09:53:26,692 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315726692 2023-08-06 09:53:26,693 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-06 09:53:26,693 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:26,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN}] 2023-08-06 09:53:26,695 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:26,695 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:26,695 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:26,943 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,943 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:26,943 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:26,943 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:26,943 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:26,943 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:26,943 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:26,943 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:26,944 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:26,944 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:26,946 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-06 09:53:26,947 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:26,947 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:26,947 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:26,948 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:26,948 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:26,949 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:26,949 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:26,949 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:26,950 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:26,950 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,951 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:26,951 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,951 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,951 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,951 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-06 09:53:26,952 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:26,952 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:26,952 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315636952 2023-08-06 09:53:26,953 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:26,954 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:26,957 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:26,957 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=733289ef46def4ea49c9f7f8372be558, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:26,958 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:26,958 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315666958 2023-08-06 09:53:26,958 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315726958 2023-08-06 09:53:26,958 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-06 09:53:26,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN}] 2023-08-06 09:53:26,958 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:26,961 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:26,962 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=733289ef46def4ea49c9f7f8372be558, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:26,962 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315606962"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315606962"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315606962"}]},"ts":"1691315606962"} 2023-08-06 09:53:26,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:26,964 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:26,964 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:26,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:26,964 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:27,208 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,208 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:27,208 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:27,208 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:27,209 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:27,209 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:27,209 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:27,209 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:27,209 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:27,210 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:27,211 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-06 09:53:27,211 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-06 09:53:27,211 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:27,212 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:27,212 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:27,212 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:27,213 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:27,213 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:27,213 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:27,213 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:27,214 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:27,215 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,215 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:27,215 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:27,215 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,215 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,215 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,215 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:27,215 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:27,215 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-06 09:53:27,216 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:27,216 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:27,217 INFO [Listener at localhost/36789] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558 to restore RIT 2023-08-06 09:53:27,217 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315637217 2023-08-06 09:53:27,218 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:27,218 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:27,220 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:27,220 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=733289ef46def4ea49c9f7f8372be558, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:27,220 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:27,220 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315667220 2023-08-06 09:53:27,220 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315727220 2023-08-06 09:53:27,220 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:53:27,221 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=733289ef46def4ea49c9f7f8372be558, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:27,221 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315607221"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315607221"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315607221"}]},"ts":"1691315607221"} 2023-08-06 09:53:27,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:27,223 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:27,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:27,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 733289ef46def4ea49c9f7f8372be558, disabling compactions & flushes 2023-08-06 09:53:27,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:27,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:27,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. after waiting 0 ms 2023-08-06 09:53:27,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:27,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 733289ef46def4ea49c9f7f8372be558 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-06 09:53:27,387 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/97fc0670742241b1b41ded7bbcba3ca7 2023-08-06 09:53:27,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/78a53babc2014ff5b4e282d20edbb40e 2023-08-06 09:53:27,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf1/97fc0670742241b1b41ded7bbcba3ca7 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7 2023-08-06 09:53:27,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:53:27,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/.tmp/cf2/78a53babc2014ff5b4e282d20edbb40e as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e 2023-08-06 09:53:27,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:53:27,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 733289ef46def4ea49c9f7f8372be558 in 46ms, sequenceid=82, compaction requested=true 2023-08-06 09:53:27,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-06 09:53:27,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:27,446 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. 2023-08-06 09:53:27,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 733289ef46def4ea49c9f7f8372be558: 2023-08-06 09:53:27,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:27,449 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=733289ef46def4ea49c9f7f8372be558, regionState=CLOSED 2023-08-06 09:53:27,449 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315607449"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315607449"}]},"ts":"1691315607449"} 2023-08-06 09:53:27,451 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:27,451 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:27,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:27,451 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:27,471 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,471 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:27,471 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:27,471 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:27,472 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:27,472 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:27,472 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:27,472 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:27,472 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:27,473 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:27,475 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-06 09:53:27,475 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-06 09:53:27,475 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-08-06 09:53:27,475 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:27,476 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:27,476 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:27,476 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:27,477 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:27,477 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:27,478 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:27,478 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:27,478 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:27,479 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,479 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:27,480 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:27,480 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:27,480 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,480 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,480 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,480 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:27,480 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:27,481 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:27,481 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-06 09:53:27,481 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:27,482 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:27,482 INFO [Listener at localhost/36789] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558 to restore RIT 2023-08-06 09:53:27,482 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315637482 2023-08-06 09:53:27,483 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:27,483 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:27,486 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:27,486 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=733289ef46def4ea49c9f7f8372be558, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:27,486 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:27,486 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315667486 2023-08-06 09:53:27,486 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315727486 2023-08-06 09:53:27,486 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:53:27,487 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=733289ef46def4ea49c9f7f8372be558, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:27,487 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315607487"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315607487"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315607487"}]},"ts":"1691315607487"} 2023-08-06 09:53:27,489 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:27,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-08-06 09:53:27,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 in 267 msec 2023-08-06 09:53:27,492 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:27,492 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:27,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:27,492 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:27,737 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,737 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:27,737 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:27,737 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:27,737 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:27,738 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:27,738 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:27,738 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:27,738 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:27,739 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:27,741 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-06 09:53:27,741 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-06 09:53:27,741 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:27,741 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:27,742 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:27,743 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:27,743 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:27,743 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:27,744 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:27,744 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:27,744 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:27,745 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:27,746 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,746 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:27,746 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:27,746 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:27,746 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,747 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,747 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,747 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:27,747 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:27,747 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-06 09:53:27,748 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:27,748 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:27,749 INFO [Listener at localhost/36789] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558 to restore RIT 2023-08-06 09:53:27,749 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315637749 2023-08-06 09:53:27,750 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:27,750 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:27,753 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:27,753 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=733289ef46def4ea49c9f7f8372be558, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:27,753 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:27,753 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315667753 2023-08-06 09:53:27,753 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315727753 2023-08-06 09:53:27,753 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:53:27,754 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:27,755 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:27,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-08-06 09:53:27,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN in 797 msec 2023-08-06 09:53:27,762 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:27,762 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:27,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:27,762 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:28,004 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,004 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:28,004 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:28,004 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:28,005 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:28,006 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:28,006 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:28,006 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:28,006 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:28,007 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:28,008 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-06 09:53:28,008 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:28,008 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:28,008 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:28,009 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:28,009 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:28,009 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:28,010 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:28,010 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:28,010 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:28,010 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:28,010 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:28,011 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,011 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:28,011 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:28,011 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:28,011 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:28,012 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:28,012 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-06 09:53:28,012 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:28,013 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:28,013 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315638013 2023-08-06 09:53:28,014 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:28,014 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,017 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:28,017 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=733289ef46def4ea49c9f7f8372be558, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:28,017 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:28,017 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315668017 2023-08-06 09:53:28,017 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315728017 2023-08-06 09:53:28,018 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:53:28,018 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,018 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,018 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,022 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:28,033 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=733289ef46def4ea49c9f7f8372be558, threads=14 2023-08-06 09:53:28,034 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,034 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,034 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,035 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,035 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,035 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,035 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,035 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,035 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,036 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,036 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,042 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,042 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,042 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,218 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,218 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,218 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,271 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,289 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,290 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,302 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,307 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,314 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,315 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,317 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,319 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,319 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,319 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,319 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 733289ef46def4ea49c9f7f8372be558 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-06 09:53:28,358 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:28,359 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:28,359 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:28,518 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,518 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:28,518 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:28,518 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:28,518 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:28,519 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:28,519 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:28,519 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:28,519 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:28,520 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:28,521 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-06 09:53:28,521 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:28,521 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:28,521 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:28,522 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:28,522 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:28,522 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:28,523 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:28,523 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:28,523 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:28,524 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:28,524 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:28,525 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,525 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:28,525 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:28,525 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:28,525 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,525 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,525 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,525 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:28,525 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:28,525 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-06 09:53:28,526 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:28,527 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:28,527 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315638527 2023-08-06 09:53:28,529 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,529 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:28,531 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:28,531 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=733289ef46def4ea49c9f7f8372be558, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:28,531 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:28,531 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315668531 2023-08-06 09:53:28,531 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315728531 2023-08-06 09:53:28,531 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:53:28,542 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=733289ef46def4ea49c9f7f8372be558, threads=14 2023-08-06 09:53:28,543 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,544 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,544 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,544 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,546 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,547 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,547 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,550 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,551 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,551 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,551 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,554 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,555 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,555 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,737 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,771 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,771 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,789 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,797 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,803 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,807 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,811 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,815 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,821 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,827 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5 for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,831 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,834 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,836 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda for region: 733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:28,836 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 733289ef46def4ea49c9f7f8372be558 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-06 09:53:28,879 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:28,885 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:53:28,887 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:53:28,887 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:28,887 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:28,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:28,887 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:29,032 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,032 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:29,032 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:29,032 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:29,033 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:29,033 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:29,033 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:29,033 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:29,033 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:29,034 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:29,035 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-06 09:53:29,035 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:29,035 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:29,036 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:29,036 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:29,036 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:29,037 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:29,037 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:29,037 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:29,038 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:29,038 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:29,038 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:29,039 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,039 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:29,039 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:29,039 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:29,039 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,039 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,039 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,039 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:29,039 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:29,039 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-06 09:53:29,040 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:29,041 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:29,044 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315639044 2023-08-06 09:53:29,045 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:29,045 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,048 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:29,048 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=733289ef46def4ea49c9f7f8372be558, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:29,049 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:29,049 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315669049 2023-08-06 09:53:29,049 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315729049 2023-08-06 09:53:29,049 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-06 09:53:29,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:29,054 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:29,054 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:29,054 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:29,153 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-06 09:53:29,299 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:29,299 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:29,299 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:29,300 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:29,300 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:29,300 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:29,300 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:29,300 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:29,300 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:29,301 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:29,302 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-06 09:53:29,302 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:29,302 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:29,302 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:29,303 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:29,303 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:29,303 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:29,303 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:29,303 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:29,304 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:29,304 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:29,304 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:29,305 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,305 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN 2023-08-06 09:53:29,305 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:29,305 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:29,305 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,305 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,305 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,305 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=733289ef46def4ea49c9f7f8372be558, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:29,305 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 733289ef46def4ea49c9f7f8372be558, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:29,306 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-06 09:53:29,306 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:29,307 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:29,307 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315639307 2023-08-06 09:53:29,307 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:29,309 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,311 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:29,311 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=733289ef46def4ea49c9f7f8372be558, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:29,311 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:29,311 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315669311 2023-08-06 09:53:29,312 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315729312 2023-08-06 09:53:29,312 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-06 09:53:29,316 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315609316"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1691315609316"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1691315609316"}]},"ts":"1691315609316"} 2023-08-06 09:53:29,316 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315609316"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315609316"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315609316"}]},"ts":"1691315609316"} 2023-08-06 09:53:29,316 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315609316"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315609316"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315609316"}]},"ts":"1691315609316"} 2023-08-06 09:53:29,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=dd6855342e402ecbf93bd85c5d1bcd99, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=01989eef194238353eaba1e7788e719d, ASSIGN}] 2023-08-06 09:53:29,324 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=01989eef194238353eaba1e7788e719d, ASSIGN 2023-08-06 09:53:29,325 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=01989eef194238353eaba1e7788e719d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44015,1691315596630; forceNewPlan=false, retain=false 2023-08-06 09:53:29,326 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=dd6855342e402ecbf93bd85c5d1bcd99, ASSIGN 2023-08-06 09:53:29,327 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=dd6855342e402ecbf93bd85c5d1bcd99, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44015,1691315596630; forceNewPlan=false, retain=false 2023-08-06 09:53:29,476 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-06 09:53:29,477 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=dd6855342e402ecbf93bd85c5d1bcd99, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:29,478 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315609477"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315609477"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315609477"}]},"ts":"1691315609477"} 2023-08-06 09:53:29,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure dd6855342e402ecbf93bd85c5d1bcd99, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:29,483 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=01989eef194238353eaba1e7788e719d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:29,484 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315609483"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315609483"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315609483"}]},"ts":"1691315609483"} 2023-08-06 09:53:29,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 01989eef194238353eaba1e7788e719d, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:29,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. 2023-08-06 09:53:29,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dd6855342e402ecbf93bd85c5d1bcd99, NAME => 'testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.', STARTKEY => '', ENDKEY => '41'} 2023-08-06 09:53:29,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:29,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:29,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:29,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:29,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:29,638 INFO [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:29,639 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1 2023-08-06 09:53:29,639 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1 2023-08-06 09:53:29,639 INFO [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd6855342e402ecbf93bd85c5d1bcd99 columnFamilyName cf1 2023-08-06 09:53:29,657 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793-bottom 2023-08-06 09:53:29,662 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5-bottom 2023-08-06 09:53:29,666 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112-bottom 2023-08-06 09:53:29,673 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7-bottom 2023-08-06 09:53:29,679 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d-bottom 2023-08-06 09:53:29,685 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9-bottom 2023-08-06 09:53:29,690 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda-bottom 2023-08-06 09:53:29,691 INFO [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(310): Store=dd6855342e402ecbf93bd85c5d1bcd99/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:29,691 INFO [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:29,692 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2 2023-08-06 09:53:29,692 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2 2023-08-06 09:53:29,692 INFO [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd6855342e402ecbf93bd85c5d1bcd99 columnFamilyName cf2 2023-08-06 09:53:29,708 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6-bottom 2023-08-06 09:53:29,712 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc-bottom 2023-08-06 09:53:29,717 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e-bottom 2023-08-06 09:53:29,722 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e-bottom 2023-08-06 09:53:29,727 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00-bottom 2023-08-06 09:53:29,731 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c-bottom 2023-08-06 09:53:29,736 DEBUG [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6-bottom 2023-08-06 09:53:29,736 INFO [StoreOpener-dd6855342e402ecbf93bd85c5d1bcd99-1] regionserver.HStore(310): Store=dd6855342e402ecbf93bd85c5d1bcd99/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:29,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:29,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:29,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:29,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:29,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dd6855342e402ecbf93bd85c5d1bcd99; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63749381, jitterRate=-0.050060197710990906}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:29,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dd6855342e402ecbf93bd85c5d1bcd99: 2023-08-06 09:53:29,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99., pid=33, masterSystemTime=1691315609632 2023-08-06 09:53:29,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:29,743 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:29,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-06 09:53:29,743 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:29,743 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. 2023-08-06 09:53:29,743 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. 2023-08-06 09:53:29,743 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1912): dd6855342e402ecbf93bd85c5d1bcd99/cf1 is initiating minor compaction (all files) 2023-08-06 09:53:29,744 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1912): dd6855342e402ecbf93bd85c5d1bcd99/cf2 is initiating minor compaction (all files) 2023-08-06 09:53:29,744 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of dd6855342e402ecbf93bd85c5d1bcd99/cf1 in testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. 2023-08-06 09:53:29,744 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2259): Starting compaction of dd6855342e402ecbf93bd85c5d1bcd99/cf2 in testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. 2023-08-06 09:53:29,744 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7-bottom] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/.tmp, totalSize=34.9 K 2023-08-06 09:53:29,744 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e-bottom] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/.tmp, totalSize=34.9 K 2023-08-06 09:53:29,745 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315604603 2023-08-06 09:53:29,745 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315604603 2023-08-06 09:53:29,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. 2023-08-06 09:53:29,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. 2023-08-06 09:53:29,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. 2023-08-06 09:53:29,746 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315604820 2023-08-06 09:53:29,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 01989eef194238353eaba1e7788e719d, NAME => 'testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.', STARTKEY => '41', ENDKEY => ''} 2023-08-06 09:53:29,746 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=dd6855342e402ecbf93bd85c5d1bcd99, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:29,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:29,746 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315604820 2023-08-06 09:53:29,746 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315609746"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315609746"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315609746"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315609746"}]},"ts":"1691315609746"} 2023-08-06 09:53:29,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,746 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315605045 2023-08-06 09:53:29,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:29,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,747 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315605045 2023-08-06 09:53:29,748 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315605260 2023-08-06 09:53:29,748 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315605260 2023-08-06 09:53:29,749 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315605468 2023-08-06 09:53:29,749 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315605468 2023-08-06 09:53:29,750 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315605681 2023-08-06 09:53:29,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-08-06 09:53:29,751 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315605681 2023-08-06 09:53:29,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure dd6855342e402ecbf93bd85c5d1bcd99, server=jenkins-hbase4.apache.org,44015,1691315596630 in 268 msec 2023-08-06 09:53:29,751 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315605882 2023-08-06 09:53:29,752 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315605882 2023-08-06 09:53:29,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=dd6855342e402ecbf93bd85c5d1bcd99, ASSIGN in 431 msec 2023-08-06 09:53:29,764 INFO [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,766 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1 2023-08-06 09:53:29,766 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1 2023-08-06 09:53:29,767 INFO [StoreOpener-01989eef194238353eaba1e7788e719d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 01989eef194238353eaba1e7788e719d columnFamilyName cf1 2023-08-06 09:53:29,768 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] throttle.PressureAwareThroughputController(145): dd6855342e402ecbf93bd85c5d1bcd99#cf2#compaction#32 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:29,771 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] throttle.PressureAwareThroughputController(145): dd6855342e402ecbf93bd85c5d1bcd99#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:29,789 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793-top 2023-08-06 09:53:29,793 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5-top 2023-08-06 09:53:29,805 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/.tmp/cf2/df1b14b0b300496e882e43cb7c9be88d as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/df1b14b0b300496e882e43cb7c9be88d 2023-08-06 09:53:29,806 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112-top 2023-08-06 09:53:29,809 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/.tmp/cf1/f8e3c4b5206241d98e06a5aa0c216e0e as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/f8e3c4b5206241d98e06a5aa0c216e0e 2023-08-06 09:53:29,813 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7-top 2023-08-06 09:53:29,816 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in dd6855342e402ecbf93bd85c5d1bcd99/cf2 of dd6855342e402ecbf93bd85c5d1bcd99 into df1b14b0b300496e882e43cb7c9be88d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:29,816 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for dd6855342e402ecbf93bd85c5d1bcd99: 2023-08-06 09:53:29,816 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99., storeName=dd6855342e402ecbf93bd85c5d1bcd99/cf2, priority=9, startTime=1691315609743; duration=0sec 2023-08-06 09:53:29,817 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:29,820 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in dd6855342e402ecbf93bd85c5d1bcd99/cf1 of dd6855342e402ecbf93bd85c5d1bcd99 into f8e3c4b5206241d98e06a5aa0c216e0e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:29,820 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for dd6855342e402ecbf93bd85c5d1bcd99: 2023-08-06 09:53:29,820 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99., storeName=dd6855342e402ecbf93bd85c5d1bcd99/cf1, priority=9, startTime=1691315609742; duration=0sec 2023-08-06 09:53:29,820 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:29,821 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d-top 2023-08-06 09:53:29,825 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9-top 2023-08-06 09:53:29,829 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda-top 2023-08-06 09:53:29,830 INFO [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(310): Store=01989eef194238353eaba1e7788e719d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:29,830 INFO [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,831 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2 2023-08-06 09:53:29,831 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2 2023-08-06 09:53:29,831 INFO [StoreOpener-01989eef194238353eaba1e7788e719d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 01989eef194238353eaba1e7788e719d columnFamilyName cf2 2023-08-06 09:53:29,846 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6-top 2023-08-06 09:53:29,850 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc-top 2023-08-06 09:53:29,853 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e-top 2023-08-06 09:53:29,857 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e-top 2023-08-06 09:53:29,861 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00-top 2023-08-06 09:53:29,866 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c-top 2023-08-06 09:53:29,869 DEBUG [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6-top 2023-08-06 09:53:29,870 INFO [StoreOpener-01989eef194238353eaba1e7788e719d-1] regionserver.HStore(310): Store=01989eef194238353eaba1e7788e719d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:29,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:29,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 01989eef194238353eaba1e7788e719d 2023-08-06 09:53:29,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 01989eef194238353eaba1e7788e719d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67633186, jitterRate=0.00781300663948059}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:29,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 01989eef194238353eaba1e7788e719d: 2023-08-06 09:53:29,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d., pid=34, masterSystemTime=1691315609632 2023-08-06 09:53:29,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:29,878 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:29,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:29,878 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:29,878 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. 2023-08-06 09:53:29,878 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1912): 01989eef194238353eaba1e7788e719d/cf1 is initiating minor compaction (all files) 2023-08-06 09:53:29,878 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. 2023-08-06 09:53:29,878 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1912): 01989eef194238353eaba1e7788e719d/cf2 is initiating minor compaction (all files) 2023-08-06 09:53:29,878 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 01989eef194238353eaba1e7788e719d/cf1 in testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. 2023-08-06 09:53:29,879 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 01989eef194238353eaba1e7788e719d/cf2 in testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. 2023-08-06 09:53:29,879 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7-top] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/.tmp, totalSize=34.9 K 2023-08-06 09:53:29,879 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e-top] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/.tmp, totalSize=34.9 K 2023-08-06 09:53:29,879 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691315604603 2023-08-06 09:53:29,880 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691315604603 2023-08-06 09:53:29,880 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691315604820 2023-08-06 09:53:29,880 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691315604820 2023-08-06 09:53:29,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. 2023-08-06 09:53:29,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. 2023-08-06 09:53:29,880 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691315605045 2023-08-06 09:53:29,881 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691315605045 2023-08-06 09:53:29,881 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=01989eef194238353eaba1e7788e719d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:29,881 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691315605260 2023-08-06 09:53:29,881 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315609881"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315609881"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315609881"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315609881"}]},"ts":"1691315609881"} 2023-08-06 09:53:29,881 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691315605260 2023-08-06 09:53:29,882 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691315605468 2023-08-06 09:53:29,882 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691315605468 2023-08-06 09:53:29,882 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691315605681 2023-08-06 09:53:29,882 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691315605681 2023-08-06 09:53:29,883 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691315605882 2023-08-06 09:53:29,883 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691315605882 2023-08-06 09:53:29,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-08-06 09:53:29,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 01989eef194238353eaba1e7788e719d, server=jenkins-hbase4.apache.org,44015,1691315596630 in 398 msec 2023-08-06 09:53:29,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-08-06 09:53:29,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=01989eef194238353eaba1e7788e719d, ASSIGN in 566 msec 2023-08-06 09:53:29,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d in 3.7400 sec 2023-08-06 09:53:29,902 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] throttle.PressureAwareThroughputController(145): 01989eef194238353eaba1e7788e719d#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:29,910 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] throttle.PressureAwareThroughputController(145): 01989eef194238353eaba1e7788e719d#cf2#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:29,931 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/.tmp/cf1/d8172f06c7d04ac7a3bc28f741a700d0 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/d8172f06c7d04ac7a3bc28f741a700d0 2023-08-06 09:53:29,938 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 01989eef194238353eaba1e7788e719d/cf1 of 01989eef194238353eaba1e7788e719d into d8172f06c7d04ac7a3bc28f741a700d0(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:29,938 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 01989eef194238353eaba1e7788e719d: 2023-08-06 09:53:29,938 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d., storeName=01989eef194238353eaba1e7788e719d/cf1, priority=9, startTime=1691315609877; duration=0sec 2023-08-06 09:53:29,938 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:29,943 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/.tmp/cf2/f0b0901b19f14c4b8fa4d08bbe02ee1a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/f0b0901b19f14c4b8fa4d08bbe02ee1a 2023-08-06 09:53:29,949 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 01989eef194238353eaba1e7788e719d/cf2 of 01989eef194238353eaba1e7788e719d into f0b0901b19f14c4b8fa4d08bbe02ee1a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:29,949 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 01989eef194238353eaba1e7788e719d: 2023-08-06 09:53:29,949 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d., storeName=01989eef194238353eaba1e7788e719d/cf2, priority=9, startTime=1691315609878; duration=0sec 2023-08-06 09:53:29,949 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:30,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44015] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:54744 deadline: 1691315670063, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558. is not online on jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:30,174 DEBUG [Listener at localhost/36789] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-06 09:53:30,182 INFO [Listener at localhost/36789] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-06 09:53:30,184 DEBUG [Listener at localhost/36789] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-06 09:53:30,199 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:30,201 INFO [Listener at localhost/36789] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-08-06 09:53:30,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-08-06 09:53:30,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:30,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-08-06 09:53:30,206 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315610206"}]},"ts":"1691315610206"} 2023-08-06 09:53:30,207 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-08-06 09:53:30,209 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-08-06 09:53:30,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=dd6855342e402ecbf93bd85c5d1bcd99, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=01989eef194238353eaba1e7788e719d, UNASSIGN}] 2023-08-06 09:53:30,211 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=01989eef194238353eaba1e7788e719d, UNASSIGN 2023-08-06 09:53:30,211 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=01989eef194238353eaba1e7788e719d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:30,212 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315610211"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315610211"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315610211"}]},"ts":"1691315610211"} 2023-08-06 09:53:30,213 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 01989eef194238353eaba1e7788e719d, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:30,215 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=dd6855342e402ecbf93bd85c5d1bcd99, UNASSIGN 2023-08-06 09:53:30,216 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=dd6855342e402ecbf93bd85c5d1bcd99, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:30,216 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315610216"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315610216"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315610216"}]},"ts":"1691315610216"} 2023-08-06 09:53:30,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure dd6855342e402ecbf93bd85c5d1bcd99, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:30,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-08-06 09:53:30,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 01989eef194238353eaba1e7788e719d 2023-08-06 09:53:30,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 01989eef194238353eaba1e7788e719d, disabling compactions & flushes 2023-08-06 09:53:30,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. 2023-08-06 09:53:30,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. 2023-08-06 09:53:30,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. after waiting 0 ms 2023-08-06 09:53:30,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. 2023-08-06 09:53:30,377 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7-top] to archive 2023-08-06 09:53:30,378 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:30,380 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,382 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,383 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,385 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,386 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,388 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,390 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,392 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e-top] to archive 2023-08-06 09:53:30,393 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:30,394 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,396 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,397 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,399 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,400 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,401 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,403 DEBUG [StoreCloser-testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-06 09:53:30,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:30,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d. 2023-08-06 09:53:30,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 01989eef194238353eaba1e7788e719d: 2023-08-06 09:53:30,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 01989eef194238353eaba1e7788e719d 2023-08-06 09:53:30,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:30,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dd6855342e402ecbf93bd85c5d1bcd99, disabling compactions & flushes 2023-08-06 09:53:30,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. 2023-08-06 09:53:30,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. 2023-08-06 09:53:30,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. after waiting 0 ms 2023-08-06 09:53:30,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. 2023-08-06 09:53:30,410 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=01989eef194238353eaba1e7788e719d, regionState=CLOSED 2023-08-06 09:53:30,410 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315610410"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315610410"}]},"ts":"1691315610410"} 2023-08-06 09:53:30,411 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7-bottom] to archive 2023-08-06 09:53:30,412 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:30,414 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-08-06 09:53:30,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 01989eef194238353eaba1e7788e719d, server=jenkins-hbase4.apache.org,44015,1691315596630 in 198 msec 2023-08-06 09:53:30,415 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=01989eef194238353eaba1e7788e719d, UNASSIGN in 204 msec 2023-08-06 09:53:30,417 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,418 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,420 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,421 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,422 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,424 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e-bottom] to archive 2023-08-06 09:53:30,425 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:30,426 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,427 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,428 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,429 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,430 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,432 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,433 DEBUG [StoreCloser-testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,436 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-06 09:53:30,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:30,437 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99. 2023-08-06 09:53:30,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dd6855342e402ecbf93bd85c5d1bcd99: 2023-08-06 09:53:30,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:30,439 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=dd6855342e402ecbf93bd85c5d1bcd99, regionState=CLOSED 2023-08-06 09:53:30,439 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315610439"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315610439"}]},"ts":"1691315610439"} 2023-08-06 09:53:30,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-08-06 09:53:30,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure dd6855342e402ecbf93bd85c5d1bcd99, server=jenkins-hbase4.apache.org,44015,1691315596630 in 224 msec 2023-08-06 09:53:30,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-08-06 09:53:30,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=dd6855342e402ecbf93bd85c5d1bcd99, UNASSIGN in 233 msec 2023-08-06 09:53:30,446 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315610446"}]},"ts":"1691315610446"} 2023-08-06 09:53:30,447 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-08-06 09:53:30,450 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-08-06 09:53:30,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 248 msec 2023-08-06 09:53:30,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-08-06 09:53:30,509 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-08-06 09:53:30,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-08-06 09:53:30,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:30,513 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:30,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-08-06 09:53:30,514 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:30,522 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,522 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:30,522 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d 2023-08-06 09:53:30,525 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/recovered.edits] 2023-08-06 09:53:30,525 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/recovered.edits] 2023-08-06 09:53:30,525 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/recovered.edits] 2023-08-06 09:53:30,528 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/f8e3c4b5206241d98e06a5aa0c216e0e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf1/f8e3c4b5206241d98e06a5aa0c216e0e 2023-08-06 09:53:30,529 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/.regioninfo to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/.regioninfo 2023-08-06 09:53:30,530 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/23e5f7c218364a329662875464135793 2023-08-06 09:53:30,531 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/2dce879a7489455cad671040392443a5 2023-08-06 09:53:30,532 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/df1b14b0b300496e882e43cb7c9be88d to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/cf2/df1b14b0b300496e882e43cb7c9be88d 2023-08-06 09:53:30,533 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,533 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/83b035add6304b7c9f4d2a5323538112 2023-08-06 09:53:30,534 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,535 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/97fc0670742241b1b41ded7bbcba3ca7 2023-08-06 09:53:30,535 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/.regioninfo to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/.regioninfo 2023-08-06 09:53:30,536 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,536 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/a4206353878444f987d0444fc34e0c7d 2023-08-06 09:53:30,538 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,538 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/aabf6e71d5fd418890cf8c1989bf21c9 2023-08-06 09:53:30,539 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/23e5f7c218364a329662875464135793.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,539 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,540 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf1/d66b09b9b2fa4b8b8451a57f452f0fda 2023-08-06 09:53:30,541 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/2dce879a7489455cad671040392443a5.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,541 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,542 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/83b035add6304b7c9f4d2a5323538112.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,543 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,544 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/2d46af24b3444a328be064a53e415ff6 2023-08-06 09:53:30,544 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/97fc0670742241b1b41ded7bbcba3ca7.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,547 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/a4206353878444f987d0444fc34e0c7d.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,547 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/45796fef012846449dbfbfb06142b1bc 2023-08-06 09:53:30,549 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,550 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/78a53babc2014ff5b4e282d20edbb40e 2023-08-06 09:53:30,551 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/aabf6e71d5fd418890cf8c1989bf21c9.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,552 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,552 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/86122f661a594f42a5ef8d7ae8b76b5e 2023-08-06 09:53:30,553 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf1/d66b09b9b2fa4b8b8451a57f452f0fda.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,554 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,555 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/c74948662c434ac3a7aa8808e41d4f00 2023-08-06 09:53:30,557 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,558 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e87856bd9e6f438f99e5f713c8cb0e8c 2023-08-06 09:53:30,559 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,559 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/2d46af24b3444a328be064a53e415ff6.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,560 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6 2023-08-06 09:53:30,561 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,562 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/45796fef012846449dbfbfb06142b1bc.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,564 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/01989eef194238353eaba1e7788e719d/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,564 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/78a53babc2014ff5b4e282d20edbb40e.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,565 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/recovered.edits/85.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558/recovered.edits/85.seqid 2023-08-06 09:53:30,566 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,567 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/86122f661a594f42a5ef8d7ae8b76b5e.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,568 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/d8172f06c7d04ac7a3bc28f741a700d0 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf1/d8172f06c7d04ac7a3bc28f741a700d0 2023-08-06 09:53:30,568 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/c74948662c434ac3a7aa8808e41d4f00.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,570 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e87856bd9e6f438f99e5f713c8cb0e8c.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,571 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/f0b0901b19f14c4b8fa4d08bbe02ee1a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/cf2/f0b0901b19f14c4b8fa4d08bbe02ee1a 2023-08-06 09:53:30,572 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/dd6855342e402ecbf93bd85c5d1bcd99/cf2/e8ffbbee2b8f4c12abe5e5eeaccbfdd6.733289ef46def4ea49c9f7f8372be558 2023-08-06 09:53:30,574 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/recovered.edits/90.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d/recovered.edits/90.seqid 2023-08-06 09:53:30,575 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/01989eef194238353eaba1e7788e719d 2023-08-06 09:53:30,575 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/recovered.edits/90.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99/recovered.edits/90.seqid 2023-08-06 09:53:30,576 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitWithoutPONR/dd6855342e402ecbf93bd85c5d1bcd99 2023-08-06 09:53:30,576 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-08-06 09:53:30,579 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:30,581 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-08-06 09:53:30,584 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-08-06 09:53:30,585 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:30,585 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-08-06 09:53:30,585 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315610585"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:30,585 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315610585"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:30,585 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315610585"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:30,588 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-06 09:53:30,588 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 733289ef46def4ea49c9f7f8372be558, NAME => 'testSplitWithoutPONR,,1691315604097.733289ef46def4ea49c9f7f8372be558.', STARTKEY => '', ENDKEY => ''}, {ENCODED => dd6855342e402ecbf93bd85c5d1bcd99, NAME => 'testSplitWithoutPONR,,1691315606150.dd6855342e402ecbf93bd85c5d1bcd99.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 01989eef194238353eaba1e7788e719d, NAME => 'testSplitWithoutPONR,41,1691315606150.01989eef194238353eaba1e7788e719d.', STARTKEY => '41', ENDKEY => ''}] 2023-08-06 09:53:30,588 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-08-06 09:53:30,588 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691315610588"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:30,590 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-08-06 09:53:30,592 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:30,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 82 msec 2023-08-06 09:53:30,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-08-06 09:53:30,615 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-08-06 09:53:30,631 INFO [Listener at localhost/36789] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=396 (was 392) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:44526 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:45556 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:45590 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:45020 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:47294 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:45596 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:47932 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x709851dd-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:37590 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:37380 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:37402 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:39750 [Waiting for operation #43] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:37352 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:47266 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:47332 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:47316 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399064790_17 at /127.0.0.1:45016 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=862 (was 824) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=275 (was 291), ProcessCount=175 (was 175), AvailableMemoryMB=2468 (was 2456) - AvailableMemoryMB LEAK? - 2023-08-06 09:53:30,645 INFO [Listener at localhost/36789] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=396, OpenFileDescriptor=862, MaxFileDescriptor=60000, SystemLoadAverage=275, ProcessCount=175, AvailableMemoryMB=2467 2023-08-06 09:53:30,645 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:30,648 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-06 09:53:30,648 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-06 09:53:30,650 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:30,651 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-08-06 09:53:30,652 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-06 09:53:30,654 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:30,654 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30 empty. 2023-08-06 09:53:30,655 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:30,655 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-08-06 09:53:30,667 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-08-06 09:53:30,668 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => daf817c4f515131ebc48eee8503ceb30, NAME => 'testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp 2023-08-06 09:53:30,677 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:30,678 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing daf817c4f515131ebc48eee8503ceb30, disabling compactions & flushes 2023-08-06 09:53:30,678 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:30,678 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:30,678 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. after waiting 0 ms 2023-08-06 09:53:30,678 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:30,678 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:30,678 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:30,681 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-08-06 09:53:30,682 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315610682"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315610682"}]},"ts":"1691315610682"} 2023-08-06 09:53:30,683 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-06 09:53:30,684 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-06 09:53:30,685 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315610684"}]},"ts":"1691315610684"} 2023-08-06 09:53:30,686 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-08-06 09:53:30,690 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:53:30,691 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:53:30,691 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:53:30,691 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:53:30,691 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:53:30,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN}] 2023-08-06 09:53:30,693 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:30,694 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44015,1691315596630; forceNewPlan=false, retain=false 2023-08-06 09:53:30,844 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:53:30,845 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=daf817c4f515131ebc48eee8503ceb30, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:30,846 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315610845"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315610845"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315610845"}]},"ts":"1691315610845"} 2023-08-06 09:53:30,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:31,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:31,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => daf817c4f515131ebc48eee8503ceb30, NAME => 'testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:53:31,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:31,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:31,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:31,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:31,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:31,006 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:31,007 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1 2023-08-06 09:53:31,007 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1 2023-08-06 09:53:31,008 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region daf817c4f515131ebc48eee8503ceb30 columnFamilyName cf1 2023-08-06 09:53:31,008 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(310): Store=daf817c4f515131ebc48eee8503ceb30/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:31,009 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:31,010 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2 2023-08-06 09:53:31,010 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2 2023-08-06 09:53:31,010 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region daf817c4f515131ebc48eee8503ceb30 columnFamilyName cf2 2023-08-06 09:53:31,011 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(310): Store=daf817c4f515131ebc48eee8503ceb30/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:31,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:31,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:31,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:31,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:31,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:53:31,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened daf817c4f515131ebc48eee8503ceb30; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73220362, jitterRate=0.09106841683387756}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:31,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:31,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30., pid=43, masterSystemTime=1691315610999 2023-08-06 09:53:31,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:31,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:31,021 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=daf817c4f515131ebc48eee8503ceb30, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315611021"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315611021"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315611021"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315611021"}]},"ts":"1691315611021"} 2023-08-06 09:53:31,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-08-06 09:53:31,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 in 175 msec 2023-08-06 09:53:31,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-08-06 09:53:31,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN in 334 msec 2023-08-06 09:53:31,028 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-06 09:53:31,028 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315611028"}]},"ts":"1691315611028"} 2023-08-06 09:53:31,029 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-08-06 09:53:31,035 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-08-06 09:53:31,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 386 msec 2023-08-06 09:53:31,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:31,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-06 09:53:31,162 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-06 09:53:31,162 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,162 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-06 09:53:31,163 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:53:31,163 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,163 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,164 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,164 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,164 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,164 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,165 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,165 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:31,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,166 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,166 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:31,166 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:31,166 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:31,166 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:31,166 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,166 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:31,166 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:31,166 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:31,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:31,168 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:31,168 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:31,168 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:31,168 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:31,168 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,168 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:31,168 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:31,168 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:31,168 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:31,168 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:31,169 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:31,169 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. started... 2023-08-06 09:53:31,170 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing daf817c4f515131ebc48eee8503ceb30 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-06 09:53:31,170 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,171 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,171 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,171 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:31,171 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,171 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:31,181 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d 2023-08-06 09:53:31,195 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/223c5a9f3a6c4abf9999878f41af4268 2023-08-06 09:53:31,199 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d 2023-08-06 09:53:31,204 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:53:31,204 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/223c5a9f3a6c4abf9999878f41af4268 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/223c5a9f3a6c4abf9999878f41af4268 2023-08-06 09:53:31,209 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/223c5a9f3a6c4abf9999878f41af4268, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:53:31,211 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for daf817c4f515131ebc48eee8503ceb30 in 41ms, sequenceid=6, compaction requested=false 2023-08-06 09:53:31,211 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-08-06 09:53:31,212 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:31,212 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:31,212 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:31,212 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:31,213 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,213 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:31,213 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:31,214 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,214 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:31,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:31,215 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,215 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:31,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:31,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:31,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:31,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-06 09:53:31,217 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:53:31,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e218741[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:31,217 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,219 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:31,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,219 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:31,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,219 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:31,219 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:31,219 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:31,219 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:31,219 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,220 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:31,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,220 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:31,220 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:31,220 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:31,222 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:31,222 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,222 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:31,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:31,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:31,222 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:31,223 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:31,223 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:31,222 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:31,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:31,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:31,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:31,227 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:53:31,227 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-06 09:53:31,227 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:31,227 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:31,227 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-06 09:53:31,227 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:31,229 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,229 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,229 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,229 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,229 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,229 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:31,230 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:31,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,229 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,230 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:31,230 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:31,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,233 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,233 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:31,234 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,238 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,238 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,238 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,238 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:31,238 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,238 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-06 09:53:31,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:31,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,238 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,238 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,239 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,239 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,239 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:31,238 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,239 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:31,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,239 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,239 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,240 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,240 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,240 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,240 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,240 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,339 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:31,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:31,356 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:31,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:31,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-06 09:53:31,358 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-06 09:53:31,358 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,358 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-06 09:53:31,358 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:53:31,359 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,359 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,360 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,360 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,360 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,360 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,361 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,361 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:31,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,362 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:31,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,362 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:31,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:31,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,362 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:31,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:31,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,362 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:31,362 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:31,362 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:31,363 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:31,363 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,363 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:31,363 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:31,363 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:31,364 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,364 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,364 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:31,366 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:31,366 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:31,366 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:31,366 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,366 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:31,366 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:31,366 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:31,366 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:31,366 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,366 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:31,366 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:31,366 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:31,368 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,368 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,369 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:31,371 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:31,372 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:31,372 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:31,372 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. started... 2023-08-06 09:53:31,372 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing daf817c4f515131ebc48eee8503ceb30 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:31,386 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/055b08677b664cae8d600a41d936ea09 2023-08-06 09:53:31,802 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/b030c726db2c4a5e853b13102545043a 2023-08-06 09:53:31,808 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/055b08677b664cae8d600a41d936ea09 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/055b08677b664cae8d600a41d936ea09 2023-08-06 09:53:31,813 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/055b08677b664cae8d600a41d936ea09, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:53:31,813 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/b030c726db2c4a5e853b13102545043a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/b030c726db2c4a5e853b13102545043a 2023-08-06 09:53:31,817 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/b030c726db2c4a5e853b13102545043a, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:53:31,818 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for daf817c4f515131ebc48eee8503ceb30 in 446ms, sequenceid=19, compaction requested=false 2023-08-06 09:53:31,818 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:31,818 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:31,818 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:31,818 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:31,818 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,819 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:31,819 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:31,821 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,821 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:31,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:31,821 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,821 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:31,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:31,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:31,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:31,823 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-06 09:53:31,823 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2100974a[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:31,823 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:53:31,823 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,824 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,824 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,824 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,825 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:31,825 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:31,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,825 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:31,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,825 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,825 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:31,825 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:31,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,825 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:31,825 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:31,825 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:31,825 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:31,825 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:31,826 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:31,826 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:31,826 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:31,831 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:31,831 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:31,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,831 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:31,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:31,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:31,831 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:31,831 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:31,832 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:31,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:31,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:31,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:31,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:31,835 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:53:31,835 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-06 09:53:31,835 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:31,835 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:31,835 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-06 09:53:31,835 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:31,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,837 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:31,837 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,838 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:31,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,840 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,840 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:31,840 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:31,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,843 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,843 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,843 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,843 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,843 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:31,843 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,843 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,843 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:31,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-06 09:53:31,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:31,843 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,843 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,844 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:31,844 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:31,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:31,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:31,844 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,844 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,844 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,844 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,844 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:31,844 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:31,844 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,844 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,944 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:31,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:31,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:31,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:31,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-06 09:53:31,963 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-06 09:53:31,963 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,963 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-06 09:53:31,963 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:53:31,963 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,963 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,966 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,966 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,966 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:31,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,966 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:31,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:31,966 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:31,967 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:31,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:31,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:31,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:31,968 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:31,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:31,968 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:31,968 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:31,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:31,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:31,969 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:31,969 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. started... 2023-08-06 09:53:31,969 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:31,969 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing daf817c4f515131ebc48eee8503ceb30 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:31,970 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,970 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,970 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:31,971 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,974 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:31,974 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:31,980 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/9b9eadb9e8e84775b4490ca11b678f73 2023-08-06 09:53:32,000 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/22d03ea9eb0d46098770461b4bbaef48 2023-08-06 09:53:32,005 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/9b9eadb9e8e84775b4490ca11b678f73 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9b9eadb9e8e84775b4490ca11b678f73 2023-08-06 09:53:32,011 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9b9eadb9e8e84775b4490ca11b678f73, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:53:32,012 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/22d03ea9eb0d46098770461b4bbaef48 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/22d03ea9eb0d46098770461b4bbaef48 2023-08-06 09:53:32,016 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/22d03ea9eb0d46098770461b4bbaef48, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:53:32,017 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for daf817c4f515131ebc48eee8503ceb30 in 48ms, sequenceid=32, compaction requested=true 2023-08-06 09:53:32,018 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:32,018 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:32,018 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:32,018 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:32,018 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,018 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:32,018 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:32,020 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,020 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,020 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,020 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:32,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-06 09:53:32,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c61f737[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:32,022 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:53:32,022 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,024 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,024 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,024 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,025 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,025 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,025 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:32,027 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,027 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,027 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,027 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,027 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,028 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,028 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,028 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,028 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,028 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:32,032 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:53:32,032 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-06 09:53:32,032 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:32,032 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:32,032 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-06 09:53:32,032 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:32,033 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,033 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,033 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,033 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,034 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,033 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,034 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,035 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,035 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,036 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,038 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,038 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,039 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,045 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,045 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,045 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,045 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,046 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,045 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-06 09:53:32,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:32,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,047 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,047 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,047 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,047 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:32,047 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:32,047 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,047 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,047 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,047 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,147 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:32,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:32,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:32,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:32,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-06 09:53:32,165 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-06 09:53:32,165 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,165 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-06 09:53:32,165 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:53:32,165 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,165 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,167 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,167 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,167 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,167 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,167 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,167 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:32,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:32,168 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,168 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,168 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:32,169 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:32,169 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,169 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:32,169 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:32,169 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. started... 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:32,169 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:32,169 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing daf817c4f515131ebc48eee8503ceb30 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:32,169 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:32,172 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,172 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,172 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,172 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:32,172 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,172 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:32,183 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/fc87367976c248d7aeab1f0fbbf59ce3 2023-08-06 09:53:32,195 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/da17035683e24f918e0272cb1abc213a 2023-08-06 09:53:32,200 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/fc87367976c248d7aeab1f0fbbf59ce3 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/fc87367976c248d7aeab1f0fbbf59ce3 2023-08-06 09:53:32,204 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/fc87367976c248d7aeab1f0fbbf59ce3, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:53:32,205 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/da17035683e24f918e0272cb1abc213a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/da17035683e24f918e0272cb1abc213a 2023-08-06 09:53:32,210 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/da17035683e24f918e0272cb1abc213a, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:53:32,211 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for daf817c4f515131ebc48eee8503ceb30 in 42ms, sequenceid=45, compaction requested=true 2023-08-06 09:53:32,211 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:32,211 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:32,211 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:32,211 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:32,211 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,211 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:32,211 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:32,213 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,213 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,214 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,214 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:32,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-06 09:53:32,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f69ea95[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:32,216 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:53:32,216 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,217 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,217 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,217 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,217 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,217 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,217 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,217 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:32,217 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,217 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:32,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,217 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,218 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,218 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,218 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:32,218 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:32,219 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,219 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,219 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,220 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,220 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,220 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,220 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,220 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,220 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,220 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:32,223 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:53:32,223 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-06 09:53:32,224 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:32,224 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:32,224 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-06 09:53:32,224 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:32,226 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,226 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,226 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,226 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,226 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,226 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,227 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,227 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,227 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,227 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,229 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,229 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,230 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,232 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,232 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,233 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,232 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-06 09:53:32,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:32,233 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,232 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,233 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,233 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,233 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:32,233 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,233 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,233 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,233 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:32,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,233 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,234 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,234 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,234 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,234 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,234 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,234 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,334 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:32,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:32,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:32,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:32,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-06 09:53:32,352 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-06 09:53:32,352 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,352 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-06 09:53:32,352 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:53:32,352 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,352 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,354 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,354 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,354 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,354 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,354 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,354 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:32,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,355 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,355 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,355 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:32,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,355 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:32,356 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:32,356 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:32,356 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:32,356 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:32,356 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:32,356 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:32,356 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:32,356 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:32,356 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:32,356 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,356 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,356 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:32,356 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:32,356 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:32,356 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:32,356 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:32,357 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:32,357 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:32,356 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:32,357 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:32,357 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,357 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. started... 2023-08-06 09:53:32,357 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:32,357 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:32,357 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:32,357 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing daf817c4f515131ebc48eee8503ceb30 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:32,358 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,360 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,360 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,360 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:32,361 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,361 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:32,367 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/08c85ce238fb475eb3ad07f56805dc6b 2023-08-06 09:53:32,380 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/ec5b401164ef4bfb92556881b08f5449 2023-08-06 09:53:32,384 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/08c85ce238fb475eb3ad07f56805dc6b as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/08c85ce238fb475eb3ad07f56805dc6b 2023-08-06 09:53:32,389 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/08c85ce238fb475eb3ad07f56805dc6b, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:53:32,390 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/ec5b401164ef4bfb92556881b08f5449 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/ec5b401164ef4bfb92556881b08f5449 2023-08-06 09:53:32,395 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/ec5b401164ef4bfb92556881b08f5449, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:53:32,396 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for daf817c4f515131ebc48eee8503ceb30 in 39ms, sequenceid=58, compaction requested=true 2023-08-06 09:53:32,396 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:32,396 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:32,396 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:32,396 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:32,396 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,396 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:32,396 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:32,398 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,398 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,398 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,398 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,400 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-06 09:53:32,400 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@dd07bb6[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:32,400 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:53:32,401 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,402 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,402 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,402 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,402 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,402 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:32,402 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,402 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,402 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:32,402 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,402 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,402 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:32,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,402 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,402 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,402 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:32,404 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,404 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,404 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,408 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,408 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,408 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,408 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,408 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,408 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,408 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:32,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:53:32,411 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-06 09:53:32,411 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:32,411 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:32,411 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-06 09:53:32,411 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:32,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,413 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,414 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,414 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,416 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,416 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,416 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,419 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,419 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,419 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,419 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,419 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-06 09:53:32,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:32,419 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,419 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,419 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,419 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,419 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,419 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,419 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:32,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,420 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:32,420 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,420 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,420 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,420 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,420 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,420 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,420 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,520 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:32,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:32,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:32,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:32,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-06 09:53:32,539 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-06 09:53:32,539 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,539 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-06 09:53:32,539 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:53:32,540 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,540 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,542 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,542 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,542 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,542 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,543 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,543 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-06 09:53:32,544 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:32,544 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:32,544 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:32,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,545 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:53:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:32,546 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:32,546 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:32,546 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. started... 2023-08-06 09:53:32,546 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing daf817c4f515131ebc48eee8503ceb30 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:32,546 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,547 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,547 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,547 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:32,548 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,548 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:32,557 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/44b6c76ecfcf45ada7cd113e89ff32bf 2023-08-06 09:53:32,573 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/e63db383ff3b469087d9be92c76e5f9c 2023-08-06 09:53:32,578 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/44b6c76ecfcf45ada7cd113e89ff32bf as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/44b6c76ecfcf45ada7cd113e89ff32bf 2023-08-06 09:53:32,583 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/44b6c76ecfcf45ada7cd113e89ff32bf, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:53:32,584 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/e63db383ff3b469087d9be92c76e5f9c as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/e63db383ff3b469087d9be92c76e5f9c 2023-08-06 09:53:32,589 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/e63db383ff3b469087d9be92c76e5f9c, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:53:32,590 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for daf817c4f515131ebc48eee8503ceb30 in 43ms, sequenceid=71, compaction requested=true 2023-08-06 09:53:32,590 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:32,590 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:32,590 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:32,590 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:32,590 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,590 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-06 09:53:32,590 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-06 09:53:32,592 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,592 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,592 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,592 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:32,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-06 09:53:32,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28cf1781[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:32,595 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:53:32,595 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,597 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,597 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,597 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,598 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,598 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,598 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,598 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,598 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:32,598 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,598 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:32,598 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:53:32,598 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-06 09:53:32,598 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:32,598 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:32,599 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,600 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,600 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,600 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,600 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,600 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,600 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,601 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-06 09:53:32,601 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,601 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-06 09:53:32,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:53:32,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:53:32,605 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-06 09:53:32,605 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:32,605 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:32,605 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-06 09:53:32,605 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:32,612 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,612 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,612 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,612 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:32,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:32,612 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,612 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,613 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:32,613 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,614 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:32,614 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:32,616 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-06 09:53:32,616 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,617 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,621 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,621 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,621 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,621 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,622 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,622 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:32,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-06 09:53:32,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:32,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,622 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,621 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,621 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:32,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,622 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:32,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:32,622 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,622 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-06 09:53:32,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:32,622 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:32,622 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,623 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:32,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:32,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:32,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:32,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-06 09:53:32,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:32,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-06 09:53:32,723 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:32,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:32,985 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-08-06 09:53:32,986 DEBUG [Listener at localhost/36789] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:32,987 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:32,988 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:32,988 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:32,988 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:32,989 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:32,989 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:32,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:32,989 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:33,237 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,237 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:33,237 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:33,237 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:33,238 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-08-06 09:53:33,238 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:33,238 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:33,238 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:33,238 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:33,239 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:33,240 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-08-06 09:53:33,241 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:33,241 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:33,241 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:33,242 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:33,242 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:33,243 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:33,243 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:33,243 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:33,243 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:33,244 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:33,244 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:33,244 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:33,244 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:33,245 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,245 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:33,245 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,245 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,245 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,245 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-06 09:53:33,246 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:33,246 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:33,247 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315643247 2023-08-06 09:53:33,248 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,249 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:33,251 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:33,252 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:33,252 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:33,252 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315673252 2023-08-06 09:53:33,252 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315733252 2023-08-06 09:53:33,252 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:53:33,253 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:33,254 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:33,255 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:33,255 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:33,503 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,503 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:33,503 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:33,503 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:33,504 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:33,504 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:33,504 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:33,504 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:33,504 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:33,505 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:33,507 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-06 09:53:33,507 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:33,508 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:33,508 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:33,508 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:33,509 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:33,509 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:33,509 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:33,510 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:33,510 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:33,511 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:33,511 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:33,511 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:33,512 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:33,512 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,512 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:33,512 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,512 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,513 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,513 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-06 09:53:33,513 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:33,514 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:33,514 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315643514 2023-08-06 09:53:33,515 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:33,516 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,518 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:33,518 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:33,518 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:33,519 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315673519 2023-08-06 09:53:33,519 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315733519 2023-08-06 09:53:33,519 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-06 09:53:33,519 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:33,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN}] 2023-08-06 09:53:33,521 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:33,521 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:33,521 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:33,769 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,769 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:33,769 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:33,769 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:33,770 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:33,770 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:33,770 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:33,771 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:33,771 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:33,771 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:33,773 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-06 09:53:33,773 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:33,773 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:33,773 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:33,774 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:33,774 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:33,774 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:33,774 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:33,775 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:33,775 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:33,775 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:33,776 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:33,776 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:33,776 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:33,777 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,777 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:33,777 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,777 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,777 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,777 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-06 09:53:33,778 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:33,778 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:33,778 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315643778 2023-08-06 09:53:33,780 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:33,780 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:33,782 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:33,782 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:33,782 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:33,782 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315673782 2023-08-06 09:53:33,782 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315733782 2023-08-06 09:53:33,782 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:53:33,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN}] 2023-08-06 09:53:33,783 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:33,784 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:33,785 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:33,785 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315613785"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315613785"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315613785"}]},"ts":"1691315613785"} 2023-08-06 09:53:33,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:33,787 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:33,787 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:33,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:33,787 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:34,033 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,033 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:34,033 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:34,033 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:34,035 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:34,035 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:34,035 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:34,035 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:34,035 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:34,036 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:34,037 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-06 09:53:34,037 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-06 09:53:34,037 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:34,038 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:34,038 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:34,038 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:34,038 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,039 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:34,039 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,039 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,039 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,040 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:34,040 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,040 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,040 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:34,041 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,041 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:34,041 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:34,041 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,041 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,041 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,041 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:34,041 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:34,041 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-06 09:53:34,042 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:34,042 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:34,043 INFO [Listener at localhost/36789] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30 to restore RIT 2023-08-06 09:53:34,043 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315644043 2023-08-06 09:53:34,044 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:34,044 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:34,046 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:34,046 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:34,046 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:34,046 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315674046 2023-08-06 09:53:34,046 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315734046 2023-08-06 09:53:34,046 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:53:34,046 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:34,047 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315614046"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315614046"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315614046"}]},"ts":"1691315614046"} 2023-08-06 09:53:34,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:34,048 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:34,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing daf817c4f515131ebc48eee8503ceb30, disabling compactions & flushes 2023-08-06 09:53:34,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:34,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:34,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. after waiting 0 ms 2023-08-06 09:53:34,202 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:34,202 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing daf817c4f515131ebc48eee8503ceb30 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-06 09:53:34,212 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/9d16526f7ced487288d9545f07c13b32 2023-08-06 09:53:34,225 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/4281f5451fc04f338b53069c56254365 2023-08-06 09:53:34,230 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/9d16526f7ced487288d9545f07c13b32 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9d16526f7ced487288d9545f07c13b32 2023-08-06 09:53:34,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9d16526f7ced487288d9545f07c13b32, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:53:34,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/4281f5451fc04f338b53069c56254365 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/4281f5451fc04f338b53069c56254365 2023-08-06 09:53:34,241 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/4281f5451fc04f338b53069c56254365, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:53:34,241 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for daf817c4f515131ebc48eee8503ceb30 in 39ms, sequenceid=82, compaction requested=true 2023-08-06 09:53:34,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-06 09:53:34,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:34,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:34,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:34,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,250 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED 2023-08-06 09:53:34,250 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315614250"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315614250"}]},"ts":"1691315614250"} 2023-08-06 09:53:34,251 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:34,251 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:34,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:34,251 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:34,296 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,297 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:34,297 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:34,297 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:34,297 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:34,297 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:34,297 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:34,297 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:34,297 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:34,298 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:34,299 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-06 09:53:34,299 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-06 09:53:34,299 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-08-06 09:53:34,299 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:34,300 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:34,300 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:34,300 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:34,300 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,301 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:34,301 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,301 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,301 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,302 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:34,302 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,302 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,302 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:34,303 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,303 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:34,303 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:34,303 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:34,303 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,303 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,303 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,303 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:34,303 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:34,303 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:34,304 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-06 09:53:34,304 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:34,305 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:34,305 INFO [Listener at localhost/36789] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30 to restore RIT 2023-08-06 09:53:34,305 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315644305 2023-08-06 09:53:34,306 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:34,306 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:34,309 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:34,309 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:34,309 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:34,309 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315674309 2023-08-06 09:53:34,309 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315734309 2023-08-06 09:53:34,310 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:53:34,310 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:34,310 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315614310"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315614310"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315614310"}]},"ts":"1691315614310"} 2023-08-06 09:53:34,312 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:34,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-08-06 09:53:34,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 in 264 msec 2023-08-06 09:53:34,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:34,316 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:34,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:34,317 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:34,560 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,560 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:34,560 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:34,560 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:34,561 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:34,561 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:34,562 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:34,562 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:34,562 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:34,563 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:34,563 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-06 09:53:34,564 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-06 09:53:34,564 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:34,564 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:34,565 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:34,565 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:34,565 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:34,565 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,566 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:34,566 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,566 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,566 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,567 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:34,567 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,567 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,567 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:34,568 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,568 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:34,568 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:34,568 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:34,568 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,568 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,569 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,569 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:34,569 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:34,569 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-06 09:53:34,569 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:34,570 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:34,570 INFO [Listener at localhost/36789] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30 to restore RIT 2023-08-06 09:53:34,570 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315644570 2023-08-06 09:53:34,571 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:34,571 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:34,573 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:34,573 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:34,573 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:34,574 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315674574 2023-08-06 09:53:34,574 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315734574 2023-08-06 09:53:34,574 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:53:34,574 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:34,575 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:34,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-08-06 09:53:34,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN in 792 msec 2023-08-06 09:53:34,581 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:34,581 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:34,582 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:34,824 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,824 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:34,824 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:34,824 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:34,824 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:34,825 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:34,825 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:34,825 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:34,825 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:34,825 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:34,826 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-06 09:53:34,827 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:34,827 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:34,827 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:34,827 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:34,827 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:34,827 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:34,828 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,828 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:34,828 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,828 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:34,829 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,829 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:34,829 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,829 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:34,829 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:34,830 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,830 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:34,830 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:34,830 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:34,830 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:34,830 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:34,830 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-06 09:53:34,831 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:34,832 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:34,832 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315644832 2023-08-06 09:53:34,834 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,834 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:34,836 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:34,836 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:34,836 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:34,836 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315674836 2023-08-06 09:53:34,837 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315734837 2023-08-06 09:53:34,837 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:53:34,837 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,837 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,837 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:34,842 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:34,851 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=daf817c4f515131ebc48eee8503ceb30, threads=14 2023-08-06 09:53:34,854 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/223c5a9f3a6c4abf9999878f41af4268 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,854 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/22d03ea9eb0d46098770461b4bbaef48 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,854 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/4281f5451fc04f338b53069c56254365 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,854 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/b030c726db2c4a5e853b13102545043a for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,854 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/da17035683e24f918e0272cb1abc213a for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,857 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/e63db383ff3b469087d9be92c76e5f9c for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,857 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/ec5b401164ef4bfb92556881b08f5449 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,857 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/08c85ce238fb475eb3ad07f56805dc6b for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,857 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/055b08677b664cae8d600a41d936ea09 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,858 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/44b6c76ecfcf45ada7cd113e89ff32bf for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,858 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9b9eadb9e8e84775b4490ca11b678f73 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,858 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9d16526f7ced487288d9545f07c13b32 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,862 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:34,863 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/fc87367976c248d7aeab1f0fbbf59ce3 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,089 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/4281f5451fc04f338b53069c56254365 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,103 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/08c85ce238fb475eb3ad07f56805dc6b for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,123 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/055b08677b664cae8d600a41d936ea09 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,127 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/44b6c76ecfcf45ada7cd113e89ff32bf for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,127 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/e63db383ff3b469087d9be92c76e5f9c for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,127 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/da17035683e24f918e0272cb1abc213a for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,137 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9d16526f7ced487288d9545f07c13b32 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,139 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/b030c726db2c4a5e853b13102545043a for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,139 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-06 09:53:35,140 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,185 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9b9eadb9e8e84775b4490ca11b678f73 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,186 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/22d03ea9eb0d46098770461b4bbaef48 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,195 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/ec5b401164ef4bfb92556881b08f5449 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,196 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/223c5a9f3a6c4abf9999878f41af4268 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,544 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/fc87367976c248d7aeab1f0fbbf59ce3 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,544 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region daf817c4f515131ebc48eee8503ceb30 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-06 09:53:35,624 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:35,624 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:35,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:35,624 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:35,851 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:35,852 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:35,852 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:35,852 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:35,853 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:35,853 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:35,853 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:35,853 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:35,854 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:35,855 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:35,855 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-06 09:53:35,855 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:35,856 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:35,856 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:35,856 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:35,856 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:35,856 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:35,857 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:35,857 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:35,857 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:35,857 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:35,857 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:35,858 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:35,858 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:35,858 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:35,858 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:35,859 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:35,859 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:35,859 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:35,859 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:35,859 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:35,859 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:35,859 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:35,859 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:35,859 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:35,859 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-06 09:53:35,860 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:35,861 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:35,861 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315645861 2023-08-06 09:53:35,862 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:35,862 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:35,865 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:35,865 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:35,865 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:35,865 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315675865 2023-08-06 09:53:35,865 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315735865 2023-08-06 09:53:35,865 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:53:35,874 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=daf817c4f515131ebc48eee8503ceb30, threads=14 2023-08-06 09:53:35,875 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/223c5a9f3a6c4abf9999878f41af4268 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,875 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/22d03ea9eb0d46098770461b4bbaef48 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,875 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/4281f5451fc04f338b53069c56254365 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,876 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/b030c726db2c4a5e853b13102545043a for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,876 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/da17035683e24f918e0272cb1abc213a for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,876 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/e63db383ff3b469087d9be92c76e5f9c for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,876 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/ec5b401164ef4bfb92556881b08f5449 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,877 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/055b08677b664cae8d600a41d936ea09 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,877 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/08c85ce238fb475eb3ad07f56805dc6b for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,877 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/44b6c76ecfcf45ada7cd113e89ff32bf for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,878 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9b9eadb9e8e84775b4490ca11b678f73 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,878 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9d16526f7ced487288d9545f07c13b32 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,878 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:35,879 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/fc87367976c248d7aeab1f0fbbf59ce3 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,030 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/ec5b401164ef4bfb92556881b08f5449 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,045 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9d16526f7ced487288d9545f07c13b32 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,046 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/4281f5451fc04f338b53069c56254365 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,058 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/055b08677b664cae8d600a41d936ea09 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,074 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/08c85ce238fb475eb3ad07f56805dc6b for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,076 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/44b6c76ecfcf45ada7cd113e89ff32bf for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,091 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/e63db383ff3b469087d9be92c76e5f9c for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,103 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/fc87367976c248d7aeab1f0fbbf59ce3 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,104 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/da17035683e24f918e0272cb1abc213a for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,115 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9b9eadb9e8e84775b4490ca11b678f73 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,117 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/223c5a9f3a6c4abf9999878f41af4268 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,119 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/b030c726db2c4a5e853b13102545043a for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,120 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,120 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/22d03ea9eb0d46098770461b4bbaef48 for region: daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:36,120 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region daf817c4f515131ebc48eee8503ceb30 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-06 09:53:36,162 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:36,169 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/1c3299eae6e5d57111ef71f146bf1e79/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:53:36,171 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/25e7cf8cc7d7a6145f823f228184303d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:53:36,171 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:36,171 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:36,171 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:36,365 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,366 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:36,366 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:36,366 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:36,367 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:36,367 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:36,367 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:36,367 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:36,368 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:36,369 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:36,369 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-06 09:53:36,370 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:36,370 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:36,370 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:36,371 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:36,371 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:36,371 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:36,371 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:36,371 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:36,372 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:36,372 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:36,372 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:36,372 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:36,372 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:36,373 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:36,373 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:36,374 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,374 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:36,374 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:36,376 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:36,376 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,376 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,376 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,377 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:36,377 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:36,377 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-06 09:53:36,378 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:36,378 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:36,382 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315646382 2023-08-06 09:53:36,383 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,383 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:36,386 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:36,386 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:36,387 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:36,387 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315676387 2023-08-06 09:53:36,387 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315736387 2023-08-06 09:53:36,387 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:53:36,391 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:36,394 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:36,394 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:36,394 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:36,638 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,638 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:36,638 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:36,638 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:36,638 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:36,638 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:36,638 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:36,638 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:36,638 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:36,639 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:36,640 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-06 09:53:36,640 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:36,640 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:36,641 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:36,641 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:36,641 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:36,641 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:36,642 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:36,642 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:36,642 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:36,642 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:36,643 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:36,643 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:36,643 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:36,643 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:36,644 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:36,644 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,645 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:36,645 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:36,645 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:36,645 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,645 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,645 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,645 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:36,645 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:36,646 DEBUG [Listener at localhost/36789] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,646 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-06 09:53:36,646 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:36,647 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:36,647 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315646647 2023-08-06 09:53:36,647 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:36,650 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,652 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:36,652 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:36,652 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:36,652 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315676652 2023-08-06 09:53:36,652 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315736652 2023-08-06 09:53:36,653 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-06 09:53:36,653 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:36,670 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:36,670 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-06 09:53:36,670 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:36,670 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:36,904 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,904 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:36,904 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:36,904 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:36,904 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:36,904 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:36,904 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:36,904 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:36,904 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:36,905 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:36,906 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-08-06 09:53:36,906 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:36,906 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:36,906 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:36,907 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:36,907 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:36,907 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:36,907 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:36,907 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:36,908 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:36,908 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:36,908 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:36,908 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:36,908 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:36,908 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:36,909 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:36,910 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,910 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:36,910 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:36,910 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:36,910 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,910 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,910 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,910 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:36,911 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:36,911 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-06 09:53:36,911 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:36,912 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:36,912 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315646912 2023-08-06 09:53:36,914 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:36,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:36,917 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:36,918 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:36,918 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:36,918 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315676918 2023-08-06 09:53:36,918 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315736918 2023-08-06 09:53:36,918 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:53:36,918 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:36,920 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:36,920 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-06 09:53:36,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:36,920 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:37,169 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,169 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:37,169 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:37,170 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:37,170 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:37,170 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:37,170 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:37,170 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:37,170 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:37,171 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:37,171 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-06 09:53:37,172 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:37,172 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:37,172 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:37,172 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:37,172 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:37,172 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:37,173 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,173 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:37,173 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,173 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,173 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,174 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:37,174 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,174 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,174 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:37,175 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,175 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:37,175 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:37,175 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:37,175 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,175 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,175 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,175 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:37,175 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:37,175 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-06 09:53:37,176 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:37,176 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:37,177 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315647177 2023-08-06 09:53:37,179 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:37,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,181 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:37,181 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:37,182 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:37,182 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315677182 2023-08-06 09:53:37,182 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315737182 2023-08-06 09:53:37,182 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:53:37,182 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:37,185 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:37,185 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-06 09:53:37,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:37,185 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:37,432 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,432 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:37,432 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:37,432 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:37,434 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:37,434 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:37,434 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:37,434 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:37,434 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:37,435 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:37,435 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-06 09:53:37,435 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:37,435 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:37,436 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:37,436 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:37,436 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:37,436 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:37,437 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,437 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:37,437 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,437 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,438 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,438 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:37,438 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,438 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,439 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:37,439 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,439 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:37,439 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:37,439 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:37,440 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,440 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,440 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,440 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:37,440 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:37,440 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-06 09:53:37,441 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:37,441 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:37,442 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315647442 2023-08-06 09:53:37,442 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:37,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,446 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:37,447 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:37,447 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:37,447 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315677447 2023-08-06 09:53:37,447 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315737447 2023-08-06 09:53:37,447 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-06 09:53:37,447 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:37,451 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN] 2023-08-06 09:53:37,451 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:37,452 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-06 09:53:37,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:37,452 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:37,697 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,697 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:37,697 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:37,697 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:37,699 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:37,699 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:37,699 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:37,699 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:37,699 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:37,700 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:37,701 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-06 09:53:37,701 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:53:37,701 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:53:37,701 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-06 09:53:37,701 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:37,701 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:37,701 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:37,702 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:37,702 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,702 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:37,702 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,702 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,703 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,703 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:37,703 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,703 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,703 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:37,704 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,704 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:37,704 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:37,704 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:37,704 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:37,705 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,705 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,705 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,705 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:37,705 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:37,705 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:37,705 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-06 09:53:37,706 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:37,706 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:37,706 INFO [Listener at localhost/36789] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30 to restore RIT 2023-08-06 09:53:37,707 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315647707 2023-08-06 09:53:37,707 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:37,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,711 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:37,713 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:37,713 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:37,713 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:37,713 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315677713 2023-08-06 09:53:37,714 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315737714 2023-08-06 09:53:37,714 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 6 msec 2023-08-06 09:53:37,714 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:37,719 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:37,719 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-06 09:53:37,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:37,719 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:37,964 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,964 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:37,964 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:37,964 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:37,965 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:37,965 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:37,965 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:37,965 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:37,965 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:37,966 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:37,967 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-06 09:53:37,967 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-06 09:53:37,967 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:37,968 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:37,968 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:37,968 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:37,968 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,968 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:37,968 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,969 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:37,969 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,969 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:37,969 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,969 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:37,970 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:37,970 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,970 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:37,970 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:37,970 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,970 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,970 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,970 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:37,971 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-06 09:53:37,971 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:37,972 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:37,972 INFO [Listener at localhost/36789] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30 to restore RIT 2023-08-06 09:53:37,972 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315647972 2023-08-06 09:53:37,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:37,974 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:37,975 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:37,977 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:37,977 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:37,977 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:37,977 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315677977 2023-08-06 09:53:37,977 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315737977 2023-08-06 09:53:37,977 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-06 09:53:37,978 INFO [PEWorker-1] procedure.MasterProcedureScheduler(718): Waiting on xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN held by pid=44 2023-08-06 09:53:37,978 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:37,978 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:37,980 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:37,980 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-06 09:53:37,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:37,980 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:38,228 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,228 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:38,228 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:38,228 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:38,228 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:38,228 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:38,228 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:38,228 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:38,228 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:38,229 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:38,230 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-06 09:53:38,230 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-06 09:53:38,230 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:38,230 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:38,231 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:38,231 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:38,231 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:38,231 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:38,231 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:38,232 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:38,232 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:38,232 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:38,232 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:38,232 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:38,232 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:38,233 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,233 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:38,233 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:38,233 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,233 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,233 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,233 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:38,234 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-06 09:53:38,234 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:38,235 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:38,235 INFO [Listener at localhost/36789] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30 to restore RIT 2023-08-06 09:53:38,237 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315648237 2023-08-06 09:53:38,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,239 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:38,240 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:38,241 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:38,242 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:38,242 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:38,242 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315678242 2023-08-06 09:53:38,242 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315738242 2023-08-06 09:53:38,242 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:53:38,242 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:38,244 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:38,244 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-06 09:53:38,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:38,245 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:38,492 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,492 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:38,492 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:38,492 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:38,495 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:38,495 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:38,495 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:38,495 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:38,495 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:38,496 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:38,496 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-06 09:53:38,497 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-06 09:53:38,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:38,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:38,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:38,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:38,498 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:38,498 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:38,498 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:38,498 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:38,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:38,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:38,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:38,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:38,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:38,500 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,500 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:38,500 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:38,500 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,500 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,500 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,500 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:53:38,500 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-06 09:53:38,501 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:38,501 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:38,501 INFO [Listener at localhost/36789] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30 to restore RIT 2023-08-06 09:53:38,504 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315648504 2023-08-06 09:53:38,504 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:38,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,508 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:38,510 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:38,510 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:38,510 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:38,510 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315678510 2023-08-06 09:53:38,510 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315738510 2023-08-06 09:53:38,510 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 6 msec 2023-08-06 09:53:38,510 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:53:38,513 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d exec-time=5.5240 sec 2023-08-06 09:53:38,513 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:38,514 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,44015,1691315596630; forceNewPlan=false, retain=true 2023-08-06 09:53:38,664 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:53:38,665 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=daf817c4f515131ebc48eee8503ceb30, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:38,666 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315618665"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315618665"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315618665"}]},"ts":"1691315618665"} 2023-08-06 09:53:38,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:38,668 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:53:38,668 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:38,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:53:38,668 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:53:38,760 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:38,761 WARN [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-08-06 09:53:38,761 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:53:38,761 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:53:38,761 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:53:38,761 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:53:38,761 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:53:38,761 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:53:38,761 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:53:38,761 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:53:38,762 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:53:38,763 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-08-06 09:53:38,763 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:53:38,764 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:53:38,764 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:53:38,764 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:53:38,764 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:38,765 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:53:38,765 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:38,765 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:53:38,765 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:38,766 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:53:38,766 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:38,766 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:53:38,767 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:38,767 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:53:38,767 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:38,767 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:53:38,767 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:38,767 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-06 09:53:38,768 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:53:38,768 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:53:38,768 INFO [Listener at localhost/36789] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30 to restore RIT 2023-08-06 09:53:38,769 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315648769 2023-08-06 09:53:38,770 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:53:38,770 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:53:38,772 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:53:38,772 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=daf817c4f515131ebc48eee8503ceb30, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:53:38,772 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:53:38,772 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315678772 2023-08-06 09:53:38,772 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315738772 2023-08-06 09:53:38,772 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:53:38,772 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=daf817c4f515131ebc48eee8503ceb30, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:38,772 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315618772"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315618772"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315618772"}]},"ts":"1691315618772"} 2023-08-06 09:53:38,774 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:38,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:38,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => daf817c4f515131ebc48eee8503ceb30, NAME => 'testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:53:38,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:38,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:38,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:38,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:38,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:38,931 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:38,932 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1 2023-08-06 09:53:38,932 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1 2023-08-06 09:53:38,933 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region daf817c4f515131ebc48eee8503ceb30 columnFamilyName cf1 2023-08-06 09:53:38,939 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/055b08677b664cae8d600a41d936ea09 2023-08-06 09:53:38,943 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/08c85ce238fb475eb3ad07f56805dc6b 2023-08-06 09:53:38,946 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/44b6c76ecfcf45ada7cd113e89ff32bf 2023-08-06 09:53:38,950 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9b9eadb9e8e84775b4490ca11b678f73 2023-08-06 09:53:38,954 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9d16526f7ced487288d9545f07c13b32 2023-08-06 09:53:38,959 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d 2023-08-06 09:53:38,963 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/fc87367976c248d7aeab1f0fbbf59ce3 2023-08-06 09:53:38,963 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(310): Store=daf817c4f515131ebc48eee8503ceb30/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:38,963 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:38,964 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2 2023-08-06 09:53:38,964 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2 2023-08-06 09:53:38,964 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region daf817c4f515131ebc48eee8503ceb30 columnFamilyName cf2 2023-08-06 09:53:38,970 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/223c5a9f3a6c4abf9999878f41af4268 2023-08-06 09:53:38,973 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/22d03ea9eb0d46098770461b4bbaef48 2023-08-06 09:53:38,976 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/4281f5451fc04f338b53069c56254365 2023-08-06 09:53:38,979 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/b030c726db2c4a5e853b13102545043a 2023-08-06 09:53:38,982 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/da17035683e24f918e0272cb1abc213a 2023-08-06 09:53:38,985 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/e63db383ff3b469087d9be92c76e5f9c 2023-08-06 09:53:38,988 DEBUG [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/ec5b401164ef4bfb92556881b08f5449 2023-08-06 09:53:38,988 INFO [StoreOpener-daf817c4f515131ebc48eee8503ceb30-1] regionserver.HStore(310): Store=daf817c4f515131ebc48eee8503ceb30/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:38,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:38,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:38,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.splits 2023-08-06 09:53:38,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:38,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:38,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened daf817c4f515131ebc48eee8503ceb30; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70799543, jitterRate=0.054995402693748474}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:38,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:38,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30., pid=48, masterSystemTime=1691315618926 2023-08-06 09:53:38,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:38,996 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:38,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-06 09:53:38,996 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:38,997 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-06 09:53:38,997 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-06 09:53:38,997 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1912): daf817c4f515131ebc48eee8503ceb30/cf2 is initiating minor compaction (all files) 2023-08-06 09:53:38,997 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1912): daf817c4f515131ebc48eee8503ceb30/cf1 is initiating minor compaction (all files) 2023-08-06 09:53:38,997 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2259): Starting compaction of daf817c4f515131ebc48eee8503ceb30/cf2 in testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:38,997 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of daf817c4f515131ebc48eee8503ceb30/cf1 in testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:38,998 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/223c5a9f3a6c4abf9999878f41af4268, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/b030c726db2c4a5e853b13102545043a, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/22d03ea9eb0d46098770461b4bbaef48, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/da17035683e24f918e0272cb1abc213a, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/ec5b401164ef4bfb92556881b08f5449, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/e63db383ff3b469087d9be92c76e5f9c, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/4281f5451fc04f338b53069c56254365] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp, totalSize=34.9 K 2023-08-06 09:53:38,998 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/055b08677b664cae8d600a41d936ea09, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9b9eadb9e8e84775b4490ca11b678f73, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/fc87367976c248d7aeab1f0fbbf59ce3, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/08c85ce238fb475eb3ad07f56805dc6b, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/44b6c76ecfcf45ada7cd113e89ff32bf, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9d16526f7ced487288d9545f07c13b32] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp, totalSize=34.9 K 2023-08-06 09:53:38,999 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 223c5a9f3a6c4abf9999878f41af4268, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315611153 2023-08-06 09:53:38,999 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting dce5d629eb6449e4b4bc20e3b18d2e6d, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315611153 2023-08-06 09:53:38,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:38,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:38,999 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting b030c726db2c4a5e853b13102545043a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315611341 2023-08-06 09:53:38,999 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 055b08677b664cae8d600a41d936ea09, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315611341 2023-08-06 09:53:39,000 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=daf817c4f515131ebc48eee8503ceb30, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:39,000 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 22d03ea9eb0d46098770461b4bbaef48, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315611946 2023-08-06 09:53:39,000 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315619000"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315619000"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315619000"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315619000"}]},"ts":"1691315619000"} 2023-08-06 09:53:39,000 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 9b9eadb9e8e84775b4490ca11b678f73, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315611946 2023-08-06 09:53:39,000 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting da17035683e24f918e0272cb1abc213a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315612149 2023-08-06 09:53:39,000 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting fc87367976c248d7aeab1f0fbbf59ce3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315612149 2023-08-06 09:53:39,001 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting ec5b401164ef4bfb92556881b08f5449, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315612336 2023-08-06 09:53:39,001 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 08c85ce238fb475eb3ad07f56805dc6b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315612336 2023-08-06 09:53:39,001 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting e63db383ff3b469087d9be92c76e5f9c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315612522 2023-08-06 09:53:39,001 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 44b6c76ecfcf45ada7cd113e89ff32bf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315612522 2023-08-06 09:53:39,002 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 4281f5451fc04f338b53069c56254365, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315612725 2023-08-06 09:53:39,002 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 9d16526f7ced487288d9545f07c13b32, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315612725 2023-08-06 09:53:39,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-08-06 09:53:39,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 in 228 msec 2023-08-06 09:53:39,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN in 1.5540 sec 2023-08-06 09:53:39,023 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-08-06 09:53:39,026 DEBUG [Listener at localhost/36789] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-08-06 09:53:39,028 INFO [Listener at localhost/36789] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-08-06 09:53:39,032 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] throttle.PressureAwareThroughputController(145): daf817c4f515131ebc48eee8503ceb30#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:39,032 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] throttle.PressureAwareThroughputController(145): daf817c4f515131ebc48eee8503ceb30#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:39,033 INFO [Listener at localhost/36789] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-08-06 09:53:39,033 INFO [Listener at localhost/36789] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-08-06 09:53:39,033 INFO [Listener at localhost/36789] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-08-06 09:53:39,065 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf2/270cd86282f748e6b82e4a9f71cc7bd0 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/270cd86282f748e6b82e4a9f71cc7bd0 2023-08-06 09:53:39,072 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in daf817c4f515131ebc48eee8503ceb30/cf2 of daf817c4f515131ebc48eee8503ceb30 into 270cd86282f748e6b82e4a9f71cc7bd0(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:39,072 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:39,072 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30., storeName=daf817c4f515131ebc48eee8503ceb30/cf2, priority=9, startTime=1691315618996; duration=0sec 2023-08-06 09:53:39,073 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:39,086 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:39,087 INFO [Listener at localhost/36789] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-08-06 09:53:39,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-08-06 09:53:39,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:39,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-08-06 09:53:39,092 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315619092"}]},"ts":"1691315619092"} 2023-08-06 09:53:39,093 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-08-06 09:53:39,094 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-08-06 09:53:39,095 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN}] 2023-08-06 09:53:39,096 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN 2023-08-06 09:53:39,096 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:39,096 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315619096"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315619096"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315619096"}]},"ts":"1691315619096"} 2023-08-06 09:53:39,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:53:39,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-08-06 09:53:39,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:39,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing daf817c4f515131ebc48eee8503ceb30, disabling compactions & flushes 2023-08-06 09:53:39,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:39,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-08-06 09:53:39,457 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/.tmp/cf1/25b3bc994725407e93b7eda1915e4155 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/25b3bc994725407e93b7eda1915e4155 2023-08-06 09:53:39,462 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in daf817c4f515131ebc48eee8503ceb30/cf1 of daf817c4f515131ebc48eee8503ceb30 into 25b3bc994725407e93b7eda1915e4155(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:39,462 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:39,462 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:39,462 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30., storeName=daf817c4f515131ebc48eee8503ceb30/cf1, priority=9, startTime=1691315618995; duration=0sec 2023-08-06 09:53:39,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:39,462 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:39,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. after waiting 0 ms 2023-08-06 09:53:39,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:39,464 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/055b08677b664cae8d600a41d936ea09, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9b9eadb9e8e84775b4490ca11b678f73, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/fc87367976c248d7aeab1f0fbbf59ce3, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/08c85ce238fb475eb3ad07f56805dc6b, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/44b6c76ecfcf45ada7cd113e89ff32bf, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9d16526f7ced487288d9545f07c13b32] to archive 2023-08-06 09:53:39,465 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:39,467 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/dce5d629eb6449e4b4bc20e3b18d2e6d 2023-08-06 09:53:39,468 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/055b08677b664cae8d600a41d936ea09 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/055b08677b664cae8d600a41d936ea09 2023-08-06 09:53:39,470 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9b9eadb9e8e84775b4490ca11b678f73 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9b9eadb9e8e84775b4490ca11b678f73 2023-08-06 09:53:39,471 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/fc87367976c248d7aeab1f0fbbf59ce3 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/fc87367976c248d7aeab1f0fbbf59ce3 2023-08-06 09:53:39,473 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/08c85ce238fb475eb3ad07f56805dc6b to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/08c85ce238fb475eb3ad07f56805dc6b 2023-08-06 09:53:39,474 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/44b6c76ecfcf45ada7cd113e89ff32bf to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/44b6c76ecfcf45ada7cd113e89ff32bf 2023-08-06 09:53:39,475 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9d16526f7ced487288d9545f07c13b32 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/9d16526f7ced487288d9545f07c13b32 2023-08-06 09:53:39,479 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/223c5a9f3a6c4abf9999878f41af4268, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/b030c726db2c4a5e853b13102545043a, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/22d03ea9eb0d46098770461b4bbaef48, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/da17035683e24f918e0272cb1abc213a, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/ec5b401164ef4bfb92556881b08f5449, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/e63db383ff3b469087d9be92c76e5f9c, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/4281f5451fc04f338b53069c56254365] to archive 2023-08-06 09:53:39,480 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:39,482 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/223c5a9f3a6c4abf9999878f41af4268 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/223c5a9f3a6c4abf9999878f41af4268 2023-08-06 09:53:39,483 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/b030c726db2c4a5e853b13102545043a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/b030c726db2c4a5e853b13102545043a 2023-08-06 09:53:39,484 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/22d03ea9eb0d46098770461b4bbaef48 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/22d03ea9eb0d46098770461b4bbaef48 2023-08-06 09:53:39,485 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/da17035683e24f918e0272cb1abc213a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/da17035683e24f918e0272cb1abc213a 2023-08-06 09:53:39,486 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/ec5b401164ef4bfb92556881b08f5449 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/ec5b401164ef4bfb92556881b08f5449 2023-08-06 09:53:39,488 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/e63db383ff3b469087d9be92c76e5f9c to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/e63db383ff3b469087d9be92c76e5f9c 2023-08-06 09:53:39,489 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/4281f5451fc04f338b53069c56254365 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/4281f5451fc04f338b53069c56254365 2023-08-06 09:53:39,493 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-06 09:53:39,493 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:39,494 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30. 2023-08-06 09:53:39,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for daf817c4f515131ebc48eee8503ceb30: 2023-08-06 09:53:39,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:39,496 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=daf817c4f515131ebc48eee8503ceb30, regionState=CLOSED 2023-08-06 09:53:39,496 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315619496"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315619496"}]},"ts":"1691315619496"} 2023-08-06 09:53:39,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-08-06 09:53:39,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure daf817c4f515131ebc48eee8503ceb30, server=jenkins-hbase4.apache.org,44015,1691315596630 in 400 msec 2023-08-06 09:53:39,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-08-06 09:53:39,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, UNASSIGN in 404 msec 2023-08-06 09:53:39,501 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315619501"}]},"ts":"1691315619501"} 2023-08-06 09:53:39,503 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-08-06 09:53:39,504 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-08-06 09:53:39,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 416 msec 2023-08-06 09:53:39,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-08-06 09:53:39,695 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-08-06 09:53:39,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-08-06 09:53:39,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:39,698 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:39,699 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:39,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-08-06 09:53:39,702 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:39,704 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/recovered.edits] 2023-08-06 09:53:39,707 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/25b3bc994725407e93b7eda1915e4155 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf1/25b3bc994725407e93b7eda1915e4155 2023-08-06 09:53:39,710 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/270cd86282f748e6b82e4a9f71cc7bd0 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/cf2/270cd86282f748e6b82e4a9f71cc7bd0 2023-08-06 09:53:39,713 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/recovered.edits/90.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30/recovered.edits/90.seqid 2023-08-06 09:53:39,713 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackAndDoubleExecution/daf817c4f515131ebc48eee8503ceb30 2023-08-06 09:53:39,713 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-08-06 09:53:39,716 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:39,718 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-08-06 09:53:39,720 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-08-06 09:53:39,721 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:39,721 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-08-06 09:53:39,721 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315619721"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:39,723 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-08-06 09:53:39,723 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => daf817c4f515131ebc48eee8503ceb30, NAME => 'testRollbackAndDoubleExecution,,1691315610649.daf817c4f515131ebc48eee8503ceb30.', STARTKEY => '', ENDKEY => ''}] 2023-08-06 09:53:39,723 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-08-06 09:53:39,723 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691315619723"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:39,724 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-08-06 09:53:39,726 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:53:39,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 30 msec 2023-08-06 09:53:39,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-08-06 09:53:39,801 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-08-06 09:53:39,815 INFO [Listener at localhost/36789] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=393 (was 396), OpenFileDescriptor=856 (was 862), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=248 (was 275), ProcessCount=175 (was 175), AvailableMemoryMB=2348 (was 2467) 2023-08-06 09:53:39,828 INFO [Listener at localhost/36789] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=393, OpenFileDescriptor=856, MaxFileDescriptor=60000, SystemLoadAverage=248, ProcessCount=175, AvailableMemoryMB=2347 2023-08-06 09:53:39,829 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:39,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-06 09:53:39,831 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-06 09:53:39,833 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:53:39,834 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-08-06 09:53:39,835 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-06 09:53:39,836 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:39,837 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7 empty. 2023-08-06 09:53:39,837 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:39,837 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-08-06 09:53:39,850 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-08-06 09:53:39,851 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => 76d3a54f2b27e12af5a7612811d713b7, NAME => 'testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp 2023-08-06 09:53:39,865 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:39,865 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing 76d3a54f2b27e12af5a7612811d713b7, disabling compactions & flushes 2023-08-06 09:53:39,865 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. 2023-08-06 09:53:39,866 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. 2023-08-06 09:53:39,866 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. after waiting 0 ms 2023-08-06 09:53:39,866 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. 2023-08-06 09:53:39,866 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. 2023-08-06 09:53:39,866 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for 76d3a54f2b27e12af5a7612811d713b7: 2023-08-06 09:53:39,868 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-08-06 09:53:39,869 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691315619869"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315619869"}]},"ts":"1691315619869"} 2023-08-06 09:53:39,871 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-06 09:53:39,872 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-06 09:53:39,872 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315619872"}]},"ts":"1691315619872"} 2023-08-06 09:53:39,873 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-08-06 09:53:39,876 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:53:39,877 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:53:39,877 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:53:39,877 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:53:39,877 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:53:39,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=76d3a54f2b27e12af5a7612811d713b7, ASSIGN}] 2023-08-06 09:53:39,879 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=76d3a54f2b27e12af5a7612811d713b7, ASSIGN 2023-08-06 09:53:39,880 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=76d3a54f2b27e12af5a7612811d713b7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39585,1691315596578; forceNewPlan=false, retain=false 2023-08-06 09:53:40,030 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:53:40,032 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=76d3a54f2b27e12af5a7612811d713b7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:40,032 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691315620032"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315620032"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315620032"}]},"ts":"1691315620032"} 2023-08-06 09:53:40,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 76d3a54f2b27e12af5a7612811d713b7, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:53:40,187 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:40,188 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-06 09:53:40,191 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53914, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-06 09:53:40,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. 2023-08-06 09:53:40,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 76d3a54f2b27e12af5a7612811d713b7, NAME => 'testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:53:40,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:40,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:40,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:40,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:40,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:40,199 INFO [StoreOpener-76d3a54f2b27e12af5a7612811d713b7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:40,200 DEBUG [StoreOpener-76d3a54f2b27e12af5a7612811d713b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7/cf1 2023-08-06 09:53:40,200 DEBUG [StoreOpener-76d3a54f2b27e12af5a7612811d713b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7/cf1 2023-08-06 09:53:40,201 INFO [StoreOpener-76d3a54f2b27e12af5a7612811d713b7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 76d3a54f2b27e12af5a7612811d713b7 columnFamilyName cf1 2023-08-06 09:53:40,201 INFO [StoreOpener-76d3a54f2b27e12af5a7612811d713b7-1] regionserver.HStore(310): Store=76d3a54f2b27e12af5a7612811d713b7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:40,201 INFO [StoreOpener-76d3a54f2b27e12af5a7612811d713b7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:40,203 DEBUG [StoreOpener-76d3a54f2b27e12af5a7612811d713b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7/cf2 2023-08-06 09:53:40,203 DEBUG [StoreOpener-76d3a54f2b27e12af5a7612811d713b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7/cf2 2023-08-06 09:53:40,204 INFO [StoreOpener-76d3a54f2b27e12af5a7612811d713b7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 76d3a54f2b27e12af5a7612811d713b7 columnFamilyName cf2 2023-08-06 09:53:40,204 INFO [StoreOpener-76d3a54f2b27e12af5a7612811d713b7-1] regionserver.HStore(310): Store=76d3a54f2b27e12af5a7612811d713b7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:40,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:40,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:40,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:40,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:40,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:53:40,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 76d3a54f2b27e12af5a7612811d713b7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62284482, jitterRate=-0.07188889384269714}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:40,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 76d3a54f2b27e12af5a7612811d713b7: 2023-08-06 09:53:40,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7., pid=55, masterSystemTime=1691315620187 2023-08-06 09:53:40,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. 2023-08-06 09:53:40,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. 2023-08-06 09:53:40,216 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=76d3a54f2b27e12af5a7612811d713b7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:40,216 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691315620216"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315620216"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315620216"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315620216"}]},"ts":"1691315620216"} 2023-08-06 09:53:40,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-08-06 09:53:40,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 76d3a54f2b27e12af5a7612811d713b7, server=jenkins-hbase4.apache.org,39585,1691315596578 in 182 msec 2023-08-06 09:53:40,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-08-06 09:53:40,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=76d3a54f2b27e12af5a7612811d713b7, ASSIGN in 341 msec 2023-08-06 09:53:40,222 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-06 09:53:40,222 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315620222"}]},"ts":"1691315620222"} 2023-08-06 09:53:40,223 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-08-06 09:53:40,226 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-08-06 09:53:40,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 394 msec 2023-08-06 09:53:40,334 DEBUG [Listener at localhost/36789] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:40,335 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,336 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,336 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,336 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=76d3a54f2b27e12af5a7612811d713b7, UNASSIGN}] 2023-08-06 09:53:40,339 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=76d3a54f2b27e12af5a7612811d713b7, UNASSIGN 2023-08-06 09:53:40,339 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=76d3a54f2b27e12af5a7612811d713b7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:40,339 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691315620339"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315620339"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315620339"}]},"ts":"1691315620339"} 2023-08-06 09:53:40,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 76d3a54f2b27e12af5a7612811d713b7, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:53:40,493 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:40,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 76d3a54f2b27e12af5a7612811d713b7, disabling compactions & flushes 2023-08-06 09:53:40,494 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. 2023-08-06 09:53:40,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. 2023-08-06 09:53:40,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. after waiting 0 ms 2023-08-06 09:53:40,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. 2023-08-06 09:53:40,498 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-08-06 09:53:40,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:40,499 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7. 2023-08-06 09:53:40,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 76d3a54f2b27e12af5a7612811d713b7: 2023-08-06 09:53:40,500 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:40,501 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=76d3a54f2b27e12af5a7612811d713b7, regionState=CLOSED 2023-08-06 09:53:40,501 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691315620501"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315620501"}]},"ts":"1691315620501"} 2023-08-06 09:53:40,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-08-06 09:53:40,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 76d3a54f2b27e12af5a7612811d713b7, server=jenkins-hbase4.apache.org,39585,1691315596578 in 161 msec 2023-08-06 09:53:40,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-08-06 09:53:40,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=76d3a54f2b27e12af5a7612811d713b7, UNASSIGN in 166 msec 2023-08-06 09:53:40,535 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-08-06 09:53:40,537 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-08-06 09:53:40,538 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1691315620538"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1691315620538"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1691315620538"}]},"ts":"1691315620538"} 2023-08-06 09:53:40,538 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315620538"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315620538"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315620538"}]},"ts":"1691315620538"} 2023-08-06 09:53:40,538 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315620538"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315620538"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315620538"}]},"ts":"1691315620538"} 2023-08-06 09:53:40,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5fb8d19bc752087bfea414e375a9b7aa, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e116f9f054275a43cbb613a92a2159b1, ASSIGN}] 2023-08-06 09:53:40,542 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e116f9f054275a43cbb613a92a2159b1, ASSIGN 2023-08-06 09:53:40,543 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e116f9f054275a43cbb613a92a2159b1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39585,1691315596578; forceNewPlan=false, retain=false 2023-08-06 09:53:40,544 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5fb8d19bc752087bfea414e375a9b7aa, ASSIGN 2023-08-06 09:53:40,544 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5fb8d19bc752087bfea414e375a9b7aa, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39585,1691315596578; forceNewPlan=false, retain=false 2023-08-06 09:53:40,693 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-06 09:53:40,695 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=5fb8d19bc752087bfea414e375a9b7aa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:40,695 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315620695"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315620695"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315620695"}]},"ts":"1691315620695"} 2023-08-06 09:53:40,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 5fb8d19bc752087bfea414e375a9b7aa, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:53:40,700 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=e116f9f054275a43cbb613a92a2159b1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:40,700 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315620700"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315620700"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315620700"}]},"ts":"1691315620700"} 2023-08-06 09:53:40,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure e116f9f054275a43cbb613a92a2159b1, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:53:40,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa. 2023-08-06 09:53:40,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5fb8d19bc752087bfea414e375a9b7aa, NAME => 'testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa.', STARTKEY => '', ENDKEY => '41'} 2023-08-06 09:53:40,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:40,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:40,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:40,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:40,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:40,853 INFO [StoreOpener-5fb8d19bc752087bfea414e375a9b7aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:40,855 DEBUG [StoreOpener-5fb8d19bc752087bfea414e375a9b7aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa/cf1 2023-08-06 09:53:40,855 DEBUG [StoreOpener-5fb8d19bc752087bfea414e375a9b7aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa/cf1 2023-08-06 09:53:40,855 INFO [StoreOpener-5fb8d19bc752087bfea414e375a9b7aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5fb8d19bc752087bfea414e375a9b7aa columnFamilyName cf1 2023-08-06 09:53:40,856 INFO [StoreOpener-5fb8d19bc752087bfea414e375a9b7aa-1] regionserver.HStore(310): Store=5fb8d19bc752087bfea414e375a9b7aa/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:40,856 INFO [StoreOpener-5fb8d19bc752087bfea414e375a9b7aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:40,857 DEBUG [StoreOpener-5fb8d19bc752087bfea414e375a9b7aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa/cf2 2023-08-06 09:53:40,857 DEBUG [StoreOpener-5fb8d19bc752087bfea414e375a9b7aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa/cf2 2023-08-06 09:53:40,857 INFO [StoreOpener-5fb8d19bc752087bfea414e375a9b7aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5fb8d19bc752087bfea414e375a9b7aa columnFamilyName cf2 2023-08-06 09:53:40,858 INFO [StoreOpener-5fb8d19bc752087bfea414e375a9b7aa-1] regionserver.HStore(310): Store=5fb8d19bc752087bfea414e375a9b7aa/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:40,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:40,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:40,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:40,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:40,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5fb8d19bc752087bfea414e375a9b7aa; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71269405, jitterRate=0.06199689209461212}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:40,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5fb8d19bc752087bfea414e375a9b7aa: 2023-08-06 09:53:40,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa., pid=61, masterSystemTime=1691315620849 2023-08-06 09:53:40,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa. 2023-08-06 09:53:40,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa. 2023-08-06 09:53:40,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1. 2023-08-06 09:53:40,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e116f9f054275a43cbb613a92a2159b1, NAME => 'testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1.', STARTKEY => '41', ENDKEY => ''} 2023-08-06 09:53:40,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:40,867 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=5fb8d19bc752087bfea414e375a9b7aa, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:40,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,867 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315620866"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315620866"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315620866"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315620866"}]},"ts":"1691315620866"} 2023-08-06 09:53:40,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:40,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,868 INFO [StoreOpener-e116f9f054275a43cbb613a92a2159b1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,870 DEBUG [StoreOpener-e116f9f054275a43cbb613a92a2159b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1/cf1 2023-08-06 09:53:40,870 DEBUG [StoreOpener-e116f9f054275a43cbb613a92a2159b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1/cf1 2023-08-06 09:53:40,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-08-06 09:53:40,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 5fb8d19bc752087bfea414e375a9b7aa, server=jenkins-hbase4.apache.org,39585,1691315596578 in 171 msec 2023-08-06 09:53:40,870 INFO [StoreOpener-e116f9f054275a43cbb613a92a2159b1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e116f9f054275a43cbb613a92a2159b1 columnFamilyName cf1 2023-08-06 09:53:40,870 INFO [StoreOpener-e116f9f054275a43cbb613a92a2159b1-1] regionserver.HStore(310): Store=e116f9f054275a43cbb613a92a2159b1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:40,871 INFO [StoreOpener-e116f9f054275a43cbb613a92a2159b1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5fb8d19bc752087bfea414e375a9b7aa, ASSIGN in 329 msec 2023-08-06 09:53:40,872 DEBUG [StoreOpener-e116f9f054275a43cbb613a92a2159b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1/cf2 2023-08-06 09:53:40,872 DEBUG [StoreOpener-e116f9f054275a43cbb613a92a2159b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1/cf2 2023-08-06 09:53:40,872 INFO [StoreOpener-e116f9f054275a43cbb613a92a2159b1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e116f9f054275a43cbb613a92a2159b1 columnFamilyName cf2 2023-08-06 09:53:40,872 INFO [StoreOpener-e116f9f054275a43cbb613a92a2159b1-1] regionserver.HStore(310): Store=e116f9f054275a43cbb613a92a2159b1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:40,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:40,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:40,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e116f9f054275a43cbb613a92a2159b1; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63673028, jitterRate=-0.051197946071624756}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:40,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e116f9f054275a43cbb613a92a2159b1: 2023-08-06 09:53:40,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1., pid=62, masterSystemTime=1691315620849 2023-08-06 09:53:40,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1. 2023-08-06 09:53:40,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1. 2023-08-06 09:53:40,880 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=e116f9f054275a43cbb613a92a2159b1, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:40,880 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315620879"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315620879"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315620879"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315620879"}]},"ts":"1691315620879"} 2023-08-06 09:53:40,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-08-06 09:53:40,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure e116f9f054275a43cbb613a92a2159b1, server=jenkins-hbase4.apache.org,39585,1691315596578 in 180 msec 2023-08-06 09:53:40,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-08-06 09:53:40,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e116f9f054275a43cbb613a92a2159b1, ASSIGN in 341 msec 2023-08-06 09:53:40,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 in 550 msec 2023-08-06 09:53:41,088 DEBUG [hconnection-0x45a8b49f-shared-pool-11] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-06 09:53:41,093 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45382, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-06 09:53:41,095 DEBUG [Listener at localhost/36789] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-06 09:53:41,098 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:41,099 INFO [Listener at localhost/36789] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-08-06 09:53:41,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-08-06 09:53:41,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:53:41,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-08-06 09:53:41,103 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315621103"}]},"ts":"1691315621103"} 2023-08-06 09:53:41,104 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-08-06 09:53:41,198 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-08-06 09:53:41,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5fb8d19bc752087bfea414e375a9b7aa, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e116f9f054275a43cbb613a92a2159b1, UNASSIGN}] 2023-08-06 09:53:41,199 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e116f9f054275a43cbb613a92a2159b1, UNASSIGN 2023-08-06 09:53:41,200 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=e116f9f054275a43cbb613a92a2159b1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:41,200 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315621200"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315621200"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315621200"}]},"ts":"1691315621200"} 2023-08-06 09:53:41,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure e116f9f054275a43cbb613a92a2159b1, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:53:41,204 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5fb8d19bc752087bfea414e375a9b7aa, UNASSIGN 2023-08-06 09:53:41,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-08-06 09:53:41,204 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=5fb8d19bc752087bfea414e375a9b7aa, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:41,204 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315621204"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315621204"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315621204"}]},"ts":"1691315621204"} 2023-08-06 09:53:41,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 5fb8d19bc752087bfea414e375a9b7aa, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:53:41,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:41,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e116f9f054275a43cbb613a92a2159b1, disabling compactions & flushes 2023-08-06 09:53:41,354 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1. 2023-08-06 09:53:41,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1. 2023-08-06 09:53:41,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1. after waiting 0 ms 2023-08-06 09:53:41,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1. 2023-08-06 09:53:41,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-08-06 09:53:41,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:41,360 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1. 2023-08-06 09:53:41,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e116f9f054275a43cbb613a92a2159b1: 2023-08-06 09:53:41,361 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:41,362 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:41,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5fb8d19bc752087bfea414e375a9b7aa, disabling compactions & flushes 2023-08-06 09:53:41,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa. 2023-08-06 09:53:41,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa. 2023-08-06 09:53:41,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa. after waiting 0 ms 2023-08-06 09:53:41,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa. 2023-08-06 09:53:41,363 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=e116f9f054275a43cbb613a92a2159b1, regionState=CLOSED 2023-08-06 09:53:41,363 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315621363"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315621363"}]},"ts":"1691315621363"} 2023-08-06 09:53:41,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-08-06 09:53:41,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure e116f9f054275a43cbb613a92a2159b1, server=jenkins-hbase4.apache.org,39585,1691315596578 in 163 msec 2023-08-06 09:53:41,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-08-06 09:53:41,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:41,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa. 2023-08-06 09:53:41,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5fb8d19bc752087bfea414e375a9b7aa: 2023-08-06 09:53:41,367 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e116f9f054275a43cbb613a92a2159b1, UNASSIGN in 167 msec 2023-08-06 09:53:41,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:41,368 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=5fb8d19bc752087bfea414e375a9b7aa, regionState=CLOSED 2023-08-06 09:53:41,368 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315621368"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315621368"}]},"ts":"1691315621368"} 2023-08-06 09:53:41,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-08-06 09:53:41,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 5fb8d19bc752087bfea414e375a9b7aa, server=jenkins-hbase4.apache.org,39585,1691315596578 in 164 msec 2023-08-06 09:53:41,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-08-06 09:53:41,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5fb8d19bc752087bfea414e375a9b7aa, UNASSIGN in 172 msec 2023-08-06 09:53:41,373 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315621373"}]},"ts":"1691315621373"} 2023-08-06 09:53:41,375 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-08-06 09:53:41,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-08-06 09:53:41,467 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-08-06 09:53:41,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 368 msec 2023-08-06 09:53:41,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-08-06 09:53:41,707 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-08-06 09:53:41,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-08-06 09:53:41,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:53:41,710 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:53:41,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-08-06 09:53:41,711 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:53:41,716 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:41,716 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:41,716 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:41,718 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1/recovered.edits] 2023-08-06 09:53:41,718 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa/recovered.edits] 2023-08-06 09:53:41,718 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7/recovered.edits] 2023-08-06 09:53:41,724 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1/recovered.edits/7.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1/recovered.edits/7.seqid 2023-08-06 09:53:41,724 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7/recovered.edits/4.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7/recovered.edits/4.seqid 2023-08-06 09:53:41,725 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa/recovered.edits/7.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa/recovered.edits/7.seqid 2023-08-06 09:53:41,725 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:53:41,725 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/76d3a54f2b27e12af5a7612811d713b7 2023-08-06 09:53:41,725 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionNoStoreFile/5fb8d19bc752087bfea414e375a9b7aa 2023-08-06 09:53:41,725 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-08-06 09:53:41,727 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:53:41,729 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-08-06 09:53:41,731 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-08-06 09:53:41,732 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:53:41,732 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-08-06 09:53:41,732 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315621732"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:41,732 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315621732"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:41,732 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315621732"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:41,733 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-06 09:53:41,733 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 76d3a54f2b27e12af5a7612811d713b7, NAME => 'testSplitTableRegionNoStoreFile,,1691315619832.76d3a54f2b27e12af5a7612811d713b7.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 5fb8d19bc752087bfea414e375a9b7aa, NAME => 'testSplitTableRegionNoStoreFile,,1691315620334.5fb8d19bc752087bfea414e375a9b7aa.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => e116f9f054275a43cbb613a92a2159b1, NAME => 'testSplitTableRegionNoStoreFile,41,1691315620334.e116f9f054275a43cbb613a92a2159b1.', STARTKEY => '41', ENDKEY => ''}] 2023-08-06 09:53:41,733 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-08-06 09:53:41,733 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691315621733"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:41,735 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-08-06 09:53:41,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-08-06 09:53:41,814 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:53:41,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 107 msec 2023-08-06 09:53:42,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-08-06 09:53:42,013 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-08-06 09:53:42,029 INFO [Listener at localhost/36789] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=403 (was 393) Potentially hanging thread: hconnection-0x45a8b49f-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-362688565_17 at /127.0.0.1:45596 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-362688565_17 at /127.0.0.1:59052 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=862 (was 856) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=248 (was 248), ProcessCount=175 (was 175), AvailableMemoryMB=2347 (was 2347) 2023-08-06 09:53:42,042 INFO [Listener at localhost/36789] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=402, OpenFileDescriptor=862, MaxFileDescriptor=60000, SystemLoadAverage=248, ProcessCount=175, AvailableMemoryMB=2346 2023-08-06 09:53:42,043 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:42,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-06 09:53:42,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-06 09:53:42,048 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,049 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-08-06 09:53:42,050 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-06 09:53:42,051 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:42,052 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae empty. 2023-08-06 09:53:42,052 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:42,052 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-08-06 09:53:42,073 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-08-06 09:53:42,075 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => c7bf488ae570a703d8494b8ebc889fae, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp 2023-08-06 09:53:42,085 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:42,085 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing c7bf488ae570a703d8494b8ebc889fae, disabling compactions & flushes 2023-08-06 09:53:42,085 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,085 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,085 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. after waiting 0 ms 2023-08-06 09:53:42,085 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,085 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,085 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:42,087 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-08-06 09:53:42,088 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691315622088"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315622088"}]},"ts":"1691315622088"} 2023-08-06 09:53:42,089 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-06 09:53:42,090 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-06 09:53:42,090 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315622090"}]},"ts":"1691315622090"} 2023-08-06 09:53:42,091 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-08-06 09:53:42,093 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:53:42,094 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:53:42,094 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:53:42,094 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:53:42,094 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:53:42,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c7bf488ae570a703d8494b8ebc889fae, ASSIGN}] 2023-08-06 09:53:42,096 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c7bf488ae570a703d8494b8ebc889fae, ASSIGN 2023-08-06 09:53:42,097 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c7bf488ae570a703d8494b8ebc889fae, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40847,1691315596668; forceNewPlan=false, retain=false 2023-08-06 09:53:42,247 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:53:42,248 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=c7bf488ae570a703d8494b8ebc889fae, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,248 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691315622248"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315622248"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315622248"}]},"ts":"1691315622248"} 2023-08-06 09:53:42,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure c7bf488ae570a703d8494b8ebc889fae, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:53:42,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c7bf488ae570a703d8494b8ebc889fae, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:53:42,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:42,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:42,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:42,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:42,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:42,410 INFO [StoreOpener-c7bf488ae570a703d8494b8ebc889fae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:42,411 DEBUG [StoreOpener-c7bf488ae570a703d8494b8ebc889fae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1 2023-08-06 09:53:42,411 DEBUG [StoreOpener-c7bf488ae570a703d8494b8ebc889fae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1 2023-08-06 09:53:42,412 INFO [StoreOpener-c7bf488ae570a703d8494b8ebc889fae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c7bf488ae570a703d8494b8ebc889fae columnFamilyName cf1 2023-08-06 09:53:42,412 INFO [StoreOpener-c7bf488ae570a703d8494b8ebc889fae-1] regionserver.HStore(310): Store=c7bf488ae570a703d8494b8ebc889fae/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:42,412 INFO [StoreOpener-c7bf488ae570a703d8494b8ebc889fae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:42,414 DEBUG [StoreOpener-c7bf488ae570a703d8494b8ebc889fae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2 2023-08-06 09:53:42,414 DEBUG [StoreOpener-c7bf488ae570a703d8494b8ebc889fae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2 2023-08-06 09:53:42,414 INFO [StoreOpener-c7bf488ae570a703d8494b8ebc889fae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c7bf488ae570a703d8494b8ebc889fae columnFamilyName cf2 2023-08-06 09:53:42,415 INFO [StoreOpener-c7bf488ae570a703d8494b8ebc889fae-1] regionserver.HStore(310): Store=c7bf488ae570a703d8494b8ebc889fae/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:42,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:42,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:42,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:42,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:42,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:53:42,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c7bf488ae570a703d8494b8ebc889fae; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59688049, jitterRate=-0.1105787605047226}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:42,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:42,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae., pid=71, masterSystemTime=1691315622401 2023-08-06 09:53:42,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,431 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=c7bf488ae570a703d8494b8ebc889fae, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,431 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691315622431"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315622431"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315622431"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315622431"}]},"ts":"1691315622431"} 2023-08-06 09:53:42,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-08-06 09:53:42,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure c7bf488ae570a703d8494b8ebc889fae, server=jenkins-hbase4.apache.org,40847,1691315596668 in 183 msec 2023-08-06 09:53:42,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-08-06 09:53:42,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c7bf488ae570a703d8494b8ebc889fae, ASSIGN in 340 msec 2023-08-06 09:53:42,437 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-06 09:53:42,437 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315622437"}]},"ts":"1691315622437"} 2023-08-06 09:53:42,438 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-08-06 09:53:42,440 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-08-06 09:53:42,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 393 msec 2023-08-06 09:53:42,550 DEBUG [Listener at localhost/36789] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-06 09:53:42,554 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49972, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-06 09:53:42,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:42,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,562 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:42,562 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,562 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-06 09:53:42,562 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-06 09:53:42,562 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,562 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,564 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,565 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,565 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,566 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:42,566 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:42,566 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,566 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,566 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:42,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,566 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:42,566 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:42,566 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:42,567 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:42,567 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,567 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:42,567 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:42,567 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:42,567 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:42,567 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:42,567 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,567 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. started... 2023-08-06 09:53:42,568 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c7bf488ae570a703d8494b8ebc889fae 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-06 09:53:42,569 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,569 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,569 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,569 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:42,570 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,570 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:42,580 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6 2023-08-06 09:53:42,595 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/8f0616390d844cb296df3bb4538143ae 2023-08-06 09:53:42,600 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6 2023-08-06 09:53:42,603 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:53:42,604 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/8f0616390d844cb296df3bb4538143ae as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae 2023-08-06 09:53:42,608 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:53:42,609 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c7bf488ae570a703d8494b8ebc889fae in 41ms, sequenceid=6, compaction requested=false 2023-08-06 09:53:42,609 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:42,609 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:42,610 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,610 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:42,610 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:42,610 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,610 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:42,610 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:42,611 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,611 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:42,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:42,612 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,612 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:42,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:42,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:42,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:42,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-06 09:53:42,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b5485e0[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:42,614 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:42,614 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,616 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,616 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,616 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,616 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,616 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:42,616 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:42,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,616 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:42,616 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:42,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,616 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:42,617 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:42,617 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:42,617 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:42,617 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:42,617 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:42,618 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,618 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:42,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:42,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:42,618 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:42,618 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:42,618 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:42,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:42,618 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:42,618 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:42,618 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:42,618 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:42,619 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:42,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:42,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:42,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:53:42,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:42,621 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-06 09:53:42,621 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:42,621 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:42,621 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,621 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:42,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,623 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:42,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:42,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:42,623 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,624 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:42,624 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:42,626 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,626 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,626 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,639 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,639 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,639 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,639 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,639 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,639 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,639 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:42,640 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:42,639 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:42,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,639 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,640 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,640 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:42,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,640 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:42,640 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,640 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,641 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,641 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,641 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,641 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,641 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,641 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,740 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:42,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:42,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:42,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:42,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,758 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:42,758 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,758 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-06 09:53:42,758 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-06 09:53:42,758 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,758 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,759 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,759 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,759 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,759 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,760 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,760 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:42,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,761 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,761 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,761 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:42,762 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:42,762 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:42,762 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:42,762 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:42,762 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:42,762 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:42,762 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,762 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. started... 2023-08-06 09:53:42,762 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c7bf488ae570a703d8494b8ebc889fae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:42,763 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,763 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,763 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,763 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,763 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:42,763 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:42,772 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/38755fc40e994630994caadd509e1987 2023-08-06 09:53:42,791 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/6b48656f275e4615a2166199f56512d6 2023-08-06 09:53:42,796 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/38755fc40e994630994caadd509e1987 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987 2023-08-06 09:53:42,801 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:53:42,802 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/6b48656f275e4615a2166199f56512d6 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6 2023-08-06 09:53:42,806 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:53:42,807 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c7bf488ae570a703d8494b8ebc889fae in 45ms, sequenceid=19, compaction requested=false 2023-08-06 09:53:42,808 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:42,808 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,808 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:42,808 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:42,808 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,808 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:42,808 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:42,813 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,813 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:42,813 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,813 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:42,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:42,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:42,815 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-06 09:53:42,815 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b09f75d[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:42,815 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:42,815 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,816 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,816 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,816 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,816 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,816 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:42,816 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:42,816 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:42,816 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:42,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,817 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:42,817 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:42,817 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:42,817 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:42,817 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:42,817 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:42,818 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,818 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:42,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:42,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:42,818 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:42,818 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:42,818 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:42,818 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:42,818 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:42,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:42,820 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:42,820 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:42,820 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:42,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:42,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:53:42,823 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:42,823 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-06 09:53:42,823 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:42,823 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:42,823 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,823 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:42,824 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,824 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,824 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,824 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,824 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:42,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:42,824 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,824 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,825 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:42,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,826 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,826 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:42,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:42,828 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,828 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,828 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,830 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,830 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,831 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:42,831 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:42,831 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:42,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,831 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:42,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:42,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:42,832 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,832 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:42,932 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:42,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:42,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:42,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:42,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,950 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:42,950 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,950 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-06 09:53:42,950 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-06 09:53:42,950 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,950 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,951 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,951 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,951 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,951 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:42,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:42,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:42,952 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:42,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,952 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:42,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,953 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:42,953 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,953 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:42,954 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,954 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:42,954 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:42,954 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,954 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:42,954 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. started... 2023-08-06 09:53:42,954 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c7bf488ae570a703d8494b8ebc889fae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:42,956 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,957 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,957 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,957 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:42,957 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:42,957 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:42,969 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/aac40dfa5c7e49c297c83bb015d4230b 2023-08-06 09:53:42,985 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/89f0026456a347e0b82ea6351430d9a6 2023-08-06 09:53:42,990 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/aac40dfa5c7e49c297c83bb015d4230b as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b 2023-08-06 09:53:42,993 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:53:42,994 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/89f0026456a347e0b82ea6351430d9a6 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6 2023-08-06 09:53:42,998 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:53:42,999 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c7bf488ae570a703d8494b8ebc889fae in 45ms, sequenceid=32, compaction requested=true 2023-08-06 09:53:42,999 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:42,999 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:42,999 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:43,000 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:43,000 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,000 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,000 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,004 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,004 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,004 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,004 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-06 09:53:43,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50a3041[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:43,006 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:43,006 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,008 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,008 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,008 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,008 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,008 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,008 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:43,008 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,008 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,008 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:43,008 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:43,008 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:43,009 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,009 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,009 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,011 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,011 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,011 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,011 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,011 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,011 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,011 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:53:43,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:43,014 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-06 09:53:43,014 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:43,015 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:43,015 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,015 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:43,016 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,016 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,016 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,016 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,016 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,016 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,016 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,017 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,017 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,018 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,019 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,020 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,020 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,022 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,022 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,022 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,022 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,022 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,022 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,023 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,023 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:43,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:43,022 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,023 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:43,023 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:43,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,023 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,023 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,023 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,023 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,024 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,024 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,024 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,024 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,024 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,123 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:43,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:43,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:43,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:43,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,139 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:43,139 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,139 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-06 09:53:43,139 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-06 09:53:43,139 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,139 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,140 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,140 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,140 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,140 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,141 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,141 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:43,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,141 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,141 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,141 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,142 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,142 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,142 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,142 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,142 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,142 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,142 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,142 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,142 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,142 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,142 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,142 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,142 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,143 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,143 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,143 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,143 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,143 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:43,143 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:43,143 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:43,143 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. started... 2023-08-06 09:53:43,143 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c7bf488ae570a703d8494b8ebc889fae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:43,145 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,145 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,145 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,146 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,147 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,147 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,154 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/9e3315554c2b431cb6e517f18b1126e1 2023-08-06 09:53:43,169 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/926e6638580649ca9fc04110695650d7 2023-08-06 09:53:43,173 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/9e3315554c2b431cb6e517f18b1126e1 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/9e3315554c2b431cb6e517f18b1126e1 2023-08-06 09:53:43,177 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/9e3315554c2b431cb6e517f18b1126e1, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:53:43,178 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/926e6638580649ca9fc04110695650d7 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/926e6638580649ca9fc04110695650d7 2023-08-06 09:53:43,182 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/926e6638580649ca9fc04110695650d7, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:53:43,183 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c7bf488ae570a703d8494b8ebc889fae in 40ms, sequenceid=45, compaction requested=true 2023-08-06 09:53:43,183 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:43,183 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:43,183 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:43,183 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:43,183 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,183 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,183 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,185 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,185 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,185 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,185 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-06 09:53:43,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55295c7e[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:43,188 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:43,188 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,189 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,189 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,189 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,190 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,190 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,190 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,190 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,190 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:43,190 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:43,190 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,190 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,190 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,190 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:43,190 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:43,193 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,193 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,193 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,193 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,193 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,193 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,193 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,193 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,194 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,194 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:53:43,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:43,197 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-06 09:53:43,197 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:43,197 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:43,197 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,197 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:43,201 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,201 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,201 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,201 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,201 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,201 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,201 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,201 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,202 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,203 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,204 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,205 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,205 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:43,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:43,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,208 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,209 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:43,208 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,209 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,209 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,209 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,209 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,209 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,209 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,309 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:43,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:43,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:43,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:43,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,324 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:43,324 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,324 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-06 09:53:43,324 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-06 09:53:43,324 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,324 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,325 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,325 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,325 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,325 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,326 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,326 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:43,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,327 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,327 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,327 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,327 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,327 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,327 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,327 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,328 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,327 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,328 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,328 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,328 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,328 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,328 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:43,328 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:43,328 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:43,328 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. started... 2023-08-06 09:53:43,328 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c7bf488ae570a703d8494b8ebc889fae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:43,330 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,331 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,331 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,331 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,331 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,331 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,338 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/b08b61c6bf054db49e18d1dde5b4c9d4 2023-08-06 09:53:43,350 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/8eb1fe5ca9e14e9a87980bd73ffa5f42 2023-08-06 09:53:43,355 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/b08b61c6bf054db49e18d1dde5b4c9d4 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/b08b61c6bf054db49e18d1dde5b4c9d4 2023-08-06 09:53:43,359 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/b08b61c6bf054db49e18d1dde5b4c9d4, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:53:43,360 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/8eb1fe5ca9e14e9a87980bd73ffa5f42 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8eb1fe5ca9e14e9a87980bd73ffa5f42 2023-08-06 09:53:43,364 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8eb1fe5ca9e14e9a87980bd73ffa5f42, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:53:43,365 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c7bf488ae570a703d8494b8ebc889fae in 37ms, sequenceid=58, compaction requested=true 2023-08-06 09:53:43,365 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:43,365 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:43,365 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:43,365 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:43,365 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,365 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,366 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,367 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,367 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,367 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,367 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,369 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-06 09:53:43,369 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a9da8be[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:43,369 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:43,369 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,371 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,371 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,371 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,371 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,371 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,371 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:43,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,371 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,371 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,371 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,371 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,371 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:43,371 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,371 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:43,371 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:43,374 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,374 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,374 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,374 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,374 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,374 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,374 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,374 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,374 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,374 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:53:43,378 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:43,378 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-06 09:53:43,378 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:43,378 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:43,378 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,378 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:43,379 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,379 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,379 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,379 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,379 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,379 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,379 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,380 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,380 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,385 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,387 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,387 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,388 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,390 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,390 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,390 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,390 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,390 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,390 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,390 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,391 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,391 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,390 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,390 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,391 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,391 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:43,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:43,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,391 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,391 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,391 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,391 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:43,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,391 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,391 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:43,391 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,492 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:43,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:43,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:43,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:43,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,506 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:43,506 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,507 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-06 09:53:43,507 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-06 09:53:43,507 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,507 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,511 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,511 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,511 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,511 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,512 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,512 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,512 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,513 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,513 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,513 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,513 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,513 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,513 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,513 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,513 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,513 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,513 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,513 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,513 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,514 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,514 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,514 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,514 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,514 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,514 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,514 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,514 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:43,514 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:43,514 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:43,514 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. started... 2023-08-06 09:53:43,514 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c7bf488ae570a703d8494b8ebc889fae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:53:43,515 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,516 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,516 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,516 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,516 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,516 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,523 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/0c5041247b1a4445a2d9a09dad5a2bf6 2023-08-06 09:53:43,534 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/ad4354de17e04428affa1d0fb5b7716d 2023-08-06 09:53:43,538 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/0c5041247b1a4445a2d9a09dad5a2bf6 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0c5041247b1a4445a2d9a09dad5a2bf6 2023-08-06 09:53:43,542 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0c5041247b1a4445a2d9a09dad5a2bf6, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:53:43,543 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/ad4354de17e04428affa1d0fb5b7716d as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ad4354de17e04428affa1d0fb5b7716d 2023-08-06 09:53:43,547 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ad4354de17e04428affa1d0fb5b7716d, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:53:43,548 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c7bf488ae570a703d8494b8ebc889fae in 34ms, sequenceid=71, compaction requested=true 2023-08-06 09:53:43,548 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:43,548 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,551 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,551 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,552 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,552 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-06 09:53:43,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e6a17af[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:43,554 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:43,554 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,555 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,555 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,555 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,555 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,555 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,555 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:43,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,555 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,555 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:43,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,555 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,556 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,556 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,556 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,556 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:43,556 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:43,557 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,557 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,557 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,557 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,557 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,557 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,558 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:53:43,562 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:43,562 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-06 09:53:43,562 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:43,562 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:43,562 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,562 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:43,563 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,563 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,564 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,564 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,564 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,565 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,567 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,567 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,567 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,569 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,569 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,569 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,570 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:43,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:43,570 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,570 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,569 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,570 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,570 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:43,570 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,570 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,570 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,570 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,571 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,670 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:43,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:43,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:43,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:43,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,686 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:43,686 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,686 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-06 09:53:43,686 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-06 09:53:43,687 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,687 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,688 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,688 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,688 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,688 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,689 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,689 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,689 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,690 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,690 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,690 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,690 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,690 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,690 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,690 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,690 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,691 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,691 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,691 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,691 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,691 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:43,691 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:43,691 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:43,691 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. started... 2023-08-06 09:53:43,691 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c7bf488ae570a703d8494b8ebc889fae 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-08-06 09:53:43,697 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,698 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,698 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,698 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,698 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,698 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,701 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/0591566a9f6745aa986b3a9b086cfc2f 2023-08-06 09:53:43,704 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0591566a9f6745aa986b3a9b086cfc2f 2023-08-06 09:53:43,712 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/8f623aeded074395a08d1a701de7f4e2 2023-08-06 09:53:43,716 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f623aeded074395a08d1a701de7f4e2 2023-08-06 09:53:43,716 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/0591566a9f6745aa986b3a9b086cfc2f as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f 2023-08-06 09:53:43,720 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0591566a9f6745aa986b3a9b086cfc2f 2023-08-06 09:53:43,720 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f, entries=9, sequenceid=83, filesize=5.1 K 2023-08-06 09:53:43,721 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/8f623aeded074395a08d1a701de7f4e2 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2 2023-08-06 09:53:43,725 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f623aeded074395a08d1a701de7f4e2 2023-08-06 09:53:43,725 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2, entries=9, sequenceid=83, filesize=5.1 K 2023-08-06 09:53:43,726 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for c7bf488ae570a703d8494b8ebc889fae in 35ms, sequenceid=83, compaction requested=true 2023-08-06 09:53:43,726 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:43,726 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,727 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,727 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,728 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,728 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,728 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,728 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-06 09:53:43,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ab6bc1b[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:43,730 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:43,730 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,731 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,731 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,731 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,731 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,731 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,732 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:43,732 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,732 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,732 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,732 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:43,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,732 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:43,732 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,732 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,732 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:43,735 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,735 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,735 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,738 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,738 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,738 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,738 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,738 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,739 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,738 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:53:43,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:43,741 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-06 09:53:43,741 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:43,741 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:43,742 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,742 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:43,743 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,743 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,743 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,743 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,743 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,743 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,744 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,744 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,744 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,745 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,746 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,746 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,747 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,749 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,749 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,749 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,749 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,749 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,749 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,749 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,749 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,749 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,750 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,749 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:43,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:43,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,750 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,749 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,750 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,750 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,750 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,750 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:43,750 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:43,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,754 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,754 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,851 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:43,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:43,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:43,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:43,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,864 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:43,864 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,864 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-06 09:53:43,864 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-06 09:53:43,864 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,864 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,866 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,866 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,866 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,866 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,866 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,866 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:43,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,868 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,868 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,868 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,868 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,868 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,868 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,868 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:43,868 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:43,868 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. started... 2023-08-06 09:53:43,869 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c7bf488ae570a703d8494b8ebc889fae 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-08-06 09:53:43,871 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,871 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,875 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,875 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,875 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,875 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,883 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/790ef4f3b74d4703bad0afb472b04e3d 2023-08-06 09:53:43,886 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 790ef4f3b74d4703bad0afb472b04e3d 2023-08-06 09:53:43,897 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/65789a4ef0c54644b57e08ef639bfb7a 2023-08-06 09:53:43,901 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65789a4ef0c54644b57e08ef639bfb7a 2023-08-06 09:53:43,902 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/790ef4f3b74d4703bad0afb472b04e3d as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/790ef4f3b74d4703bad0afb472b04e3d 2023-08-06 09:53:43,906 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 790ef4f3b74d4703bad0afb472b04e3d 2023-08-06 09:53:43,906 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/790ef4f3b74d4703bad0afb472b04e3d, entries=5, sequenceid=91, filesize=5.0 K 2023-08-06 09:53:43,907 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/65789a4ef0c54644b57e08ef639bfb7a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/65789a4ef0c54644b57e08ef639bfb7a 2023-08-06 09:53:43,911 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65789a4ef0c54644b57e08ef639bfb7a 2023-08-06 09:53:43,911 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/65789a4ef0c54644b57e08ef639bfb7a, entries=5, sequenceid=91, filesize=5.0 K 2023-08-06 09:53:43,912 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for c7bf488ae570a703d8494b8ebc889fae in 43ms, sequenceid=91, compaction requested=true 2023-08-06 09:53:43,912 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:43,913 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:43,914 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,914 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,915 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,915 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:43,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-06 09:53:43,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1563cb9f[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:43,916 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:43,917 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,918 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,918 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,918 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,918 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,918 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,918 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,918 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,918 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,918 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:43,918 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:43,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,918 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:43,918 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:43,918 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:43,918 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:43,919 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,919 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,919 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,919 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,920 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,920 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,921 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:43,921 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,921 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,921 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:43,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:53:43,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:43,925 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-06 09:53:43,925 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:43,925 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:43,925 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,925 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:43,926 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,926 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,926 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,926 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:43,926 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,926 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,927 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:43,927 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,928 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:43,928 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:43,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,930 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,930 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,930 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,935 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,935 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,935 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,935 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,935 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:43,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,935 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,935 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,935 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,935 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:43,936 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:43,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:43,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:43,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:43,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:43,936 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,936 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,936 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:43,936 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,936 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:43,936 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:43,936 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:43,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:43,936 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:43,937 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:43,937 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,037 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:44,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:44,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:53:44,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:53:44,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,052 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:44,052 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:44,052 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-06 09:53:44,052 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-06 09:53:44,053 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,053 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,055 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:44,055 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:44,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:44,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:44,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:44,055 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:44,055 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:44,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:44,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:44,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,056 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,056 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:53:44,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:44,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,056 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:44,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:53:44,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:44,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:44,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:44,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:44,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:44,057 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:44,058 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:53:44,058 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:53:44,058 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:44,058 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. started... 2023-08-06 09:53:44,058 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c7bf488ae570a703d8494b8ebc889fae 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-08-06 09:53:44,059 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,059 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,059 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,059 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:44,059 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,060 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:44,067 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/edb3aa3b3c7e4f92b389cef33c59412a 2023-08-06 09:53:44,072 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for edb3aa3b3c7e4f92b389cef33c59412a 2023-08-06 09:53:44,081 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/315e517a144c4e82a30f12f8c52bc7d3 2023-08-06 09:53:44,085 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 315e517a144c4e82a30f12f8c52bc7d3 2023-08-06 09:53:44,086 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/edb3aa3b3c7e4f92b389cef33c59412a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/edb3aa3b3c7e4f92b389cef33c59412a 2023-08-06 09:53:44,090 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for edb3aa3b3c7e4f92b389cef33c59412a 2023-08-06 09:53:44,090 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/edb3aa3b3c7e4f92b389cef33c59412a, entries=5, sequenceid=99, filesize=5.0 K 2023-08-06 09:53:44,090 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/315e517a144c4e82a30f12f8c52bc7d3 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/315e517a144c4e82a30f12f8c52bc7d3 2023-08-06 09:53:44,094 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 315e517a144c4e82a30f12f8c52bc7d3 2023-08-06 09:53:44,094 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/315e517a144c4e82a30f12f8c52bc7d3, entries=5, sequenceid=99, filesize=5.0 K 2023-08-06 09:53:44,094 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for c7bf488ae570a703d8494b8ebc889fae in 36ms, sequenceid=99, compaction requested=true 2023-08-06 09:53:44,095 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:44,095 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:44,095 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:53:44,095 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:53:44,095 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,095 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-06 09:53:44,095 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-06 09:53:44,096 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,096 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:44,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:44,097 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,097 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:53:44,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:44,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:44,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:44,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:44,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:44,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-06 09:53:44,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62cc59d5[Count = 0] remaining members to acquire global barrier 2023-08-06 09:53:44,098 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-06 09:53:44,098 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,101 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,101 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,101 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,101 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:44,101 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,101 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:53:44,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,101 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:44,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,101 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:53:44,101 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:44,101 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:44,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,101 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:53:44,101 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-06 09:53:44,101 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-06 09:53:44,101 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:53:44,103 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:44,103 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,103 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:44,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:44,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:44,103 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:44,103 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:44,103 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-06 09:53:44,103 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:44,103 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:44,103 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:44,103 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-06 09:53:44,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:44,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:44,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:44,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:44,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:44,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:53:44,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:53:44,106 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-06 09:53:44,106 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:53:44,106 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:53:44,106 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,106 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:53:44,107 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,107 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:53:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:53:44,107 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:44,107 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,107 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,108 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,108 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,108 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:53:44,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:44,108 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:44,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:44,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,109 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:53:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:44,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:44,110 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:53:44,111 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,111 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:44,113 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:44,113 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,113 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:44,113 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:44,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:44,113 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:53:44,113 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:44,113 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:53:44,113 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:44,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:44,113 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:44,113 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:53:44,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-06 09:53:44,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:53:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:44,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:53:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:53:44,114 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:44,114 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:53:44,114 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,114 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:53:44,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:44,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:53:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:44,114 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:53:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:53:44,115 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,115 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:44,115 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,115 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,115 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:44,214 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:53:44,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:53:44,218 DEBUG [Listener at localhost/36789] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,219 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:44,220 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:44,220 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:44,220 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:44,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c7bf488ae570a703d8494b8ebc889fae, UNASSIGN}] 2023-08-06 09:53:44,222 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c7bf488ae570a703d8494b8ebc889fae, UNASSIGN 2023-08-06 09:53:44,223 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=c7bf488ae570a703d8494b8ebc889fae, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,223 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691315624223"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315624223"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315624223"}]},"ts":"1691315624223"} 2023-08-06 09:53:44,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure c7bf488ae570a703d8494b8ebc889fae, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:53:44,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c7bf488ae570a703d8494b8ebc889fae, disabling compactions & flushes 2023-08-06 09:53:44,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:44,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:44,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. after waiting 0 ms 2023-08-06 09:53:44,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:44,377 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing c7bf488ae570a703d8494b8ebc889fae 2/2 column families, dataSize=50 B heapSize=720 B 2023-08-06 09:53:44,386 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/dcc3df610e7a494f913b769400d29bde 2023-08-06 09:53:44,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dcc3df610e7a494f913b769400d29bde 2023-08-06 09:53:44,403 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/ecbcccb9f84842c08bc61a62bfe7751e 2023-08-06 09:53:44,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ecbcccb9f84842c08bc61a62bfe7751e 2023-08-06 09:53:44,408 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf1/dcc3df610e7a494f913b769400d29bde as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/dcc3df610e7a494f913b769400d29bde 2023-08-06 09:53:44,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dcc3df610e7a494f913b769400d29bde 2023-08-06 09:53:44,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/dcc3df610e7a494f913b769400d29bde, entries=1, sequenceid=103, filesize=4.9 K 2023-08-06 09:53:44,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.tmp/cf2/ecbcccb9f84842c08bc61a62bfe7751e as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ecbcccb9f84842c08bc61a62bfe7751e 2023-08-06 09:53:44,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ecbcccb9f84842c08bc61a62bfe7751e 2023-08-06 09:53:44,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ecbcccb9f84842c08bc61a62bfe7751e, entries=1, sequenceid=103, filesize=4.9 K 2023-08-06 09:53:44,417 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for c7bf488ae570a703d8494b8ebc889fae in 40ms, sequenceid=103, compaction requested=true 2023-08-06 09:53:44,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-08-06 09:53:44,422 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:44,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae. 2023-08-06 09:53:44,422 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c7bf488ae570a703d8494b8ebc889fae: 2023-08-06 09:53:44,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,424 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=c7bf488ae570a703d8494b8ebc889fae, regionState=CLOSED 2023-08-06 09:53:44,424 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691315624424"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315624424"}]},"ts":"1691315624424"} 2023-08-06 09:53:44,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-08-06 09:53:44,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure c7bf488ae570a703d8494b8ebc889fae, server=jenkins-hbase4.apache.org,40847,1691315596668 in 201 msec 2023-08-06 09:53:44,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-08-06 09:53:44,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c7bf488ae570a703d8494b8ebc889fae, UNASSIGN in 205 msec 2023-08-06 09:53:44,432 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-06 09:53:44,443 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=c7bf488ae570a703d8494b8ebc889fae, threads=16 2023-08-06 09:53:44,444 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/315e517a144c4e82a30f12f8c52bc7d3 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,446 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/65789a4ef0c54644b57e08ef639bfb7a for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,447 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,447 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,447 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8eb1fe5ca9e14e9a87980bd73ffa5f42 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,447 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,447 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,450 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 315e517a144c4e82a30f12f8c52bc7d3 2023-08-06 09:53:44,450 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/926e6638580649ca9fc04110695650d7 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,450 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ad4354de17e04428affa1d0fb5b7716d for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,450 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ecbcccb9f84842c08bc61a62bfe7751e for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,451 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,451 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0c5041247b1a4445a2d9a09dad5a2bf6 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,452 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,458 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,459 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/790ef4f3b74d4703bad0afb472b04e3d for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,459 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/9e3315554c2b431cb6e517f18b1126e1 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,462 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65789a4ef0c54644b57e08ef639bfb7a 2023-08-06 09:53:44,466 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8eb1fe5ca9e14e9a87980bd73ffa5f42, top=false 2023-08-06 09:53:44,466 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/9e3315554c2b431cb6e517f18b1126e1, top=false 2023-08-06 09:53:44,469 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 790ef4f3b74d4703bad0afb472b04e3d 2023-08-06 09:53:44,472 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f623aeded074395a08d1a701de7f4e2 2023-08-06 09:53:44,473 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-8eb1fe5ca9e14e9a87980bd73ffa5f42 for child: b6ad8f165ee195bcbbe04c83a68f9d95, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,478 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8eb1fe5ca9e14e9a87980bd73ffa5f42 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,478 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,492 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/926e6638580649ca9fc04110695650d7, top=false 2023-08-06 09:53:44,493 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0591566a9f6745aa986b3a9b086cfc2f 2023-08-06 09:53:44,504 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65789a4ef0c54644b57e08ef639bfb7a 2023-08-06 09:53:44,504 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/65789a4ef0c54644b57e08ef639bfb7a, top=true 2023-08-06 09:53:44,506 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ecbcccb9f84842c08bc61a62bfe7751e 2023-08-06 09:53:44,506 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 790ef4f3b74d4703bad0afb472b04e3d 2023-08-06 09:53:44,506 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/790ef4f3b74d4703bad0afb472b04e3d, top=true 2023-08-06 09:53:44,511 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 315e517a144c4e82a30f12f8c52bc7d3 2023-08-06 09:53:44,511 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/315e517a144c4e82a30f12f8c52bc7d3, top=true 2023-08-06 09:53:44,512 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ad4354de17e04428affa1d0fb5b7716d, top=false 2023-08-06 09:53:44,512 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0c5041247b1a4445a2d9a09dad5a2bf6, top=false 2023-08-06 09:53:44,512 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ecbcccb9f84842c08bc61a62bfe7751e 2023-08-06 09:53:44,512 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ecbcccb9f84842c08bc61a62bfe7751e, top=true 2023-08-06 09:53:44,542 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-926e6638580649ca9fc04110695650d7 for child: b6ad8f165ee195bcbbe04c83a68f9d95, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,567 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-65789a4ef0c54644b57e08ef639bfb7a for child: c8dd2ff746c9d5cec67b26ba8e7dcef4, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,567 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/65789a4ef0c54644b57e08ef639bfb7a for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,567 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/b08b61c6bf054db49e18d1dde5b4c9d4 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,568 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-9e3315554c2b431cb6e517f18b1126e1 for child: b6ad8f165ee195bcbbe04c83a68f9d95, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,568 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-790ef4f3b74d4703bad0afb472b04e3d for child: c8dd2ff746c9d5cec67b26ba8e7dcef4, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,569 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/790ef4f3b74d4703bad0afb472b04e3d for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,569 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/dcc3df610e7a494f913b769400d29bde for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,571 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f623aeded074395a08d1a701de7f4e2 2023-08-06 09:53:44,577 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/926e6638580649ca9fc04110695650d7 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,577 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/edb3aa3b3c7e4f92b389cef33c59412a for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,579 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/b08b61c6bf054db49e18d1dde5b4c9d4, top=false 2023-08-06 09:53:44,580 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/9e3315554c2b431cb6e517f18b1126e1 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,580 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dcc3df610e7a494f913b769400d29bde 2023-08-06 09:53:44,583 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for edb3aa3b3c7e4f92b389cef33c59412a 2023-08-06 09:53:44,583 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dcc3df610e7a494f913b769400d29bde 2023-08-06 09:53:44,583 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/dcc3df610e7a494f913b769400d29bde, top=true 2023-08-06 09:53:44,587 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for edb3aa3b3c7e4f92b389cef33c59412a 2023-08-06 09:53:44,587 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/edb3aa3b3c7e4f92b389cef33c59412a, top=true 2023-08-06 09:53:44,590 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ad4354de17e04428affa1d0fb5b7716d for child: b6ad8f165ee195bcbbe04c83a68f9d95, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,604 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ecbcccb9f84842c08bc61a62bfe7751e for child: c8dd2ff746c9d5cec67b26ba8e7dcef4, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,604 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ecbcccb9f84842c08bc61a62bfe7751e for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,605 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-0c5041247b1a4445a2d9a09dad5a2bf6 for child: b6ad8f165ee195bcbbe04c83a68f9d95, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,615 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-315e517a144c4e82a30f12f8c52bc7d3 for child: c8dd2ff746c9d5cec67b26ba8e7dcef4, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,615 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/315e517a144c4e82a30f12f8c52bc7d3 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,616 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ad4354de17e04428affa1d0fb5b7716d for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,622 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0c5041247b1a4445a2d9a09dad5a2bf6 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,631 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-b08b61c6bf054db49e18d1dde5b4c9d4 for child: b6ad8f165ee195bcbbe04c83a68f9d95, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,632 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-edb3aa3b3c7e4f92b389cef33c59412a for child: c8dd2ff746c9d5cec67b26ba8e7dcef4, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,632 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/edb3aa3b3c7e4f92b389cef33c59412a for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,636 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/b08b61c6bf054db49e18d1dde5b4c9d4 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,639 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,641 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/.splits/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-dcc3df610e7a494f913b769400d29bde for child: c8dd2ff746c9d5cec67b26ba8e7dcef4, parent: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,641 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/dcc3df610e7a494f913b769400d29bde for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,641 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,643 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0591566a9f6745aa986b3a9b086cfc2f 2023-08-06 09:53:44,647 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,654 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,656 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,657 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,659 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6 for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,661 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f for region: c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:44,661 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region c7bf488ae570a703d8494b8ebc889fae Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-06 09:53:44,697 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-08-06 09:53:44,698 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-08-06 09:53:44,700 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1691315624700"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1691315624700"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1691315624700"}]},"ts":"1691315624700"} 2023-08-06 09:53:44,700 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691315624700"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315624700"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315624700"}]},"ts":"1691315624700"} 2023-08-06 09:53:44,700 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691315624700"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315624700"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315624700"}]},"ts":"1691315624700"} 2023-08-06 09:53:44,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b6ad8f165ee195bcbbe04c83a68f9d95, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c8dd2ff746c9d5cec67b26ba8e7dcef4, ASSIGN}] 2023-08-06 09:53:44,706 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c8dd2ff746c9d5cec67b26ba8e7dcef4, ASSIGN 2023-08-06 09:53:44,707 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c8dd2ff746c9d5cec67b26ba8e7dcef4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40847,1691315596668; forceNewPlan=false, retain=false 2023-08-06 09:53:44,707 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b6ad8f165ee195bcbbe04c83a68f9d95, ASSIGN 2023-08-06 09:53:44,708 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b6ad8f165ee195bcbbe04c83a68f9d95, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40847,1691315596668; forceNewPlan=false, retain=false 2023-08-06 09:53:44,857 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-06 09:53:44,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=c8dd2ff746c9d5cec67b26ba8e7dcef4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691315624859"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315624859"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315624859"}]},"ts":"1691315624859"} 2023-08-06 09:53:44,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure c8dd2ff746c9d5cec67b26ba8e7dcef4, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:53:44,863 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=b6ad8f165ee195bcbbe04c83a68f9d95, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:44,863 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691315624863"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315624863"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315624863"}]},"ts":"1691315624863"} 2023-08-06 09:53:44,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure b6ad8f165ee195bcbbe04c83a68f9d95, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:53:45,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b6ad8f165ee195bcbbe04c83a68f9d95, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.', STARTKEY => '', ENDKEY => '60'} 2023-08-06 09:53:45,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:45,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:45,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,017 INFO [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,018 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1 2023-08-06 09:53:45,018 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1 2023-08-06 09:53:45,018 INFO [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b6ad8f165ee195bcbbe04c83a68f9d95 columnFamilyName cf1 2023-08-06 09:53:45,029 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,030 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f-bottom 2023-08-06 09:53:45,034 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987-bottom 2023-08-06 09:53:45,039 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6-bottom 2023-08-06 09:53:45,043 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b-bottom 2023-08-06 09:53:45,046 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-0c5041247b1a4445a2d9a09dad5a2bf6 2023-08-06 09:53:45,049 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-9e3315554c2b431cb6e517f18b1126e1 2023-08-06 09:53:45,052 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-b08b61c6bf054db49e18d1dde5b4c9d4 2023-08-06 09:53:45,053 INFO [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(310): Store=b6ad8f165ee195bcbbe04c83a68f9d95/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:45,053 INFO [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,054 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2 2023-08-06 09:53:45,054 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2 2023-08-06 09:53:45,054 INFO [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b6ad8f165ee195bcbbe04c83a68f9d95 columnFamilyName cf2 2023-08-06 09:53:45,066 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6-bottom 2023-08-06 09:53:45,072 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6-bottom 2023-08-06 09:53:45,078 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae-bottom 2023-08-06 09:53:45,082 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,084 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2-bottom 2023-08-06 09:53:45,087 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-8eb1fe5ca9e14e9a87980bd73ffa5f42 2023-08-06 09:53:45,091 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-926e6638580649ca9fc04110695650d7 2023-08-06 09:53:45,094 DEBUG [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ad4354de17e04428affa1d0fb5b7716d 2023-08-06 09:53:45,095 INFO [StoreOpener-b6ad8f165ee195bcbbe04c83a68f9d95-1] regionserver.HStore(310): Store=b6ad8f165ee195bcbbe04c83a68f9d95/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:45,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:45,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b6ad8f165ee195bcbbe04c83a68f9d95; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69687059, jitterRate=0.03841809928417206}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:45,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b6ad8f165ee195bcbbe04c83a68f9d95: 2023-08-06 09:53:45,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95., pid=78, masterSystemTime=1691315625012 2023-08-06 09:53:45,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:45,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-06 09:53:45,102 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:45,102 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:45,103 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,103 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1912): b6ad8f165ee195bcbbe04c83a68f9d95/cf1 is initiating minor compaction (all files) 2023-08-06 09:53:45,103 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,103 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b6ad8f165ee195bcbbe04c83a68f9d95/cf1 in testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,103 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1912): b6ad8f165ee195bcbbe04c83a68f9d95/cf2 is initiating minor compaction (all files) 2023-08-06 09:53:45,103 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2259): Starting compaction of b6ad8f165ee195bcbbe04c83a68f9d95/cf2 in testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,103 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-9e3315554c2b431cb6e517f18b1126e1, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-b08b61c6bf054db49e18d1dde5b4c9d4, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-0c5041247b1a4445a2d9a09dad5a2bf6, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f-bottom] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/.tmp, totalSize=35.0 K 2023-08-06 09:53:45,103 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-926e6638580649ca9fc04110695650d7, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-8eb1fe5ca9e14e9a87980bd73ffa5f42, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ad4354de17e04428affa1d0fb5b7716d, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2-bottom] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/.tmp, totalSize=35.0 K 2023-08-06 09:53:45,104 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315622554 2023-08-06 09:53:45,104 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315622554 2023-08-06 09:53:45,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c8dd2ff746c9d5cec67b26ba8e7dcef4, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.', STARTKEY => '60', ENDKEY => ''} 2023-08-06 09:53:45,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:45,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:45,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,105 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315622742 2023-08-06 09:53:45,105 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315622742 2023-08-06 09:53:45,106 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=b6ad8f165ee195bcbbe04c83a68f9d95, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:45,106 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691315625106"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315625106"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315625106"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315625106"}]},"ts":"1691315625106"} 2023-08-06 09:53:45,106 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315622934 2023-08-06 09:53:45,106 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315622934 2023-08-06 09:53:45,107 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-926e6638580649ca9fc04110695650d7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315623125 2023-08-06 09:53:45,107 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-9e3315554c2b431cb6e517f18b1126e1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315623125 2023-08-06 09:53:45,108 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-b08b61c6bf054db49e18d1dde5b4c9d4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315623311 2023-08-06 09:53:45,107 INFO [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,107 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-8eb1fe5ca9e14e9a87980bd73ffa5f42, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315623311 2023-08-06 09:53:45,108 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-0c5041247b1a4445a2d9a09dad5a2bf6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315623493 2023-08-06 09:53:45,109 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ad4354de17e04428affa1d0fb5b7716d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315623493 2023-08-06 09:53:45,110 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1 2023-08-06 09:53:45,110 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691315623672 2023-08-06 09:53:45,110 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1 2023-08-06 09:53:45,110 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691315623672 2023-08-06 09:53:45,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-08-06 09:53:45,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure b6ad8f165ee195bcbbe04c83a68f9d95, server=jenkins-hbase4.apache.org,40847,1691315596668 in 244 msec 2023-08-06 09:53:45,111 INFO [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c8dd2ff746c9d5cec67b26ba8e7dcef4 columnFamilyName cf1 2023-08-06 09:53:45,115 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b6ad8f165ee195bcbbe04c83a68f9d95, ASSIGN in 408 msec 2023-08-06 09:53:45,128 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,129 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f-top 2023-08-06 09:53:45,135 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] throttle.PressureAwareThroughputController(145): b6ad8f165ee195bcbbe04c83a68f9d95#cf2#compaction#72 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:45,137 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987-top 2023-08-06 09:53:45,141 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6-top 2023-08-06 09:53:45,145 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b-top 2023-08-06 09:53:45,147 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] throttle.PressureAwareThroughputController(145): b6ad8f165ee195bcbbe04c83a68f9d95#cf1#compaction#73 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:45,149 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 790ef4f3b74d4703bad0afb472b04e3d 2023-08-06 09:53:45,149 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-790ef4f3b74d4703bad0afb472b04e3d 2023-08-06 09:53:45,153 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dcc3df610e7a494f913b769400d29bde 2023-08-06 09:53:45,153 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-dcc3df610e7a494f913b769400d29bde 2023-08-06 09:53:45,157 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for edb3aa3b3c7e4f92b389cef33c59412a 2023-08-06 09:53:45,157 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-edb3aa3b3c7e4f92b389cef33c59412a 2023-08-06 09:53:45,159 INFO [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(310): Store=c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:45,159 INFO [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,160 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2 2023-08-06 09:53:45,160 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2 2023-08-06 09:53:45,161 INFO [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c8dd2ff746c9d5cec67b26ba8e7dcef4 columnFamilyName cf2 2023-08-06 09:53:45,168 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/.tmp/cf2/feb6a81fb81b4480936aaedfe8fab6bf as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/feb6a81fb81b4480936aaedfe8fab6bf 2023-08-06 09:53:45,179 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b6ad8f165ee195bcbbe04c83a68f9d95/cf2 of b6ad8f165ee195bcbbe04c83a68f9d95 into feb6a81fb81b4480936aaedfe8fab6bf(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:45,179 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for b6ad8f165ee195bcbbe04c83a68f9d95: 2023-08-06 09:53:45,180 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95., storeName=b6ad8f165ee195bcbbe04c83a68f9d95/cf2, priority=9, startTime=1691315625101; duration=0sec 2023-08-06 09:53:45,180 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:45,183 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6-top 2023-08-06 09:53:45,187 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6-top 2023-08-06 09:53:45,188 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/.tmp/cf1/2cbf6f0f1c7249fb8285dc29516b8b1b as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/2cbf6f0f1c7249fb8285dc29516b8b1b 2023-08-06 09:53:45,192 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae-top 2023-08-06 09:53:45,194 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b6ad8f165ee195bcbbe04c83a68f9d95/cf1 of b6ad8f165ee195bcbbe04c83a68f9d95 into 2cbf6f0f1c7249fb8285dc29516b8b1b(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:45,194 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b6ad8f165ee195bcbbe04c83a68f9d95: 2023-08-06 09:53:45,194 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95., storeName=b6ad8f165ee195bcbbe04c83a68f9d95/cf1, priority=9, startTime=1691315625101; duration=0sec 2023-08-06 09:53:45,194 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:45,195 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,198 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2-top 2023-08-06 09:53:45,199 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 315e517a144c4e82a30f12f8c52bc7d3 2023-08-06 09:53:45,199 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-315e517a144c4e82a30f12f8c52bc7d3 2023-08-06 09:53:45,202 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65789a4ef0c54644b57e08ef639bfb7a 2023-08-06 09:53:45,202 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-65789a4ef0c54644b57e08ef639bfb7a 2023-08-06 09:53:45,207 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ecbcccb9f84842c08bc61a62bfe7751e 2023-08-06 09:53:45,207 DEBUG [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ecbcccb9f84842c08bc61a62bfe7751e 2023-08-06 09:53:45,207 INFO [StoreOpener-c8dd2ff746c9d5cec67b26ba8e7dcef4-1] regionserver.HStore(310): Store=c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:45,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:53:45,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c8dd2ff746c9d5cec67b26ba8e7dcef4; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72474930, jitterRate=0.07996061444282532}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:53:45,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c8dd2ff746c9d5cec67b26ba8e7dcef4: 2023-08-06 09:53:45,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4., pid=77, masterSystemTime=1691315625012 2023-08-06 09:53:45,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:45,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:45,215 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:45,215 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:53:45,216 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,216 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1912): c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2 is initiating minor compaction (all files) 2023-08-06 09:53:45,216 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,216 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2259): Starting compaction of c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,216 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1912): c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1 is initiating minor compaction (all files) 2023-08-06 09:53:45,216 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,216 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-65789a4ef0c54644b57e08ef639bfb7a, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-315e517a144c4e82a30f12f8c52bc7d3, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ecbcccb9f84842c08bc61a62bfe7751e] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/.tmp, totalSize=34.8 K 2023-08-06 09:53:45,216 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-790ef4f3b74d4703bad0afb472b04e3d, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-edb3aa3b3c7e4f92b389cef33c59412a, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-dcc3df610e7a494f913b769400d29bde] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/.tmp, totalSize=34.8 K 2023-08-06 09:53:45,217 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691315622554 2023-08-06 09:53:45,217 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691315622554 2023-08-06 09:53:45,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,217 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691315622742 2023-08-06 09:53:45,217 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=c8dd2ff746c9d5cec67b26ba8e7dcef4, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:45,217 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691315622742 2023-08-06 09:53:45,218 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691315622934 2023-08-06 09:53:45,219 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691315622934 2023-08-06 09:53:45,219 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1691315623672 2023-08-06 09:53:45,219 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1691315623672 2023-08-06 09:53:45,219 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-65789a4ef0c54644b57e08ef639bfb7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-08-06 09:53:45,218 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691315625217"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315625217"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315625217"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315625217"}]},"ts":"1691315625217"} 2023-08-06 09:53:45,219 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-790ef4f3b74d4703bad0afb472b04e3d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-08-06 09:53:45,220 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-315e517a144c4e82a30f12f8c52bc7d3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-08-06 09:53:45,220 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ecbcccb9f84842c08bc61a62bfe7751e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-08-06 09:53:45,220 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-edb3aa3b3c7e4f92b389cef33c59412a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-08-06 09:53:45,220 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-dcc3df610e7a494f913b769400d29bde, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-08-06 09:53:45,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-08-06 09:53:45,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure c8dd2ff746c9d5cec67b26ba8e7dcef4, server=jenkins-hbase4.apache.org,40847,1691315596668 in 360 msec 2023-08-06 09:53:45,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-08-06 09:53:45,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c8dd2ff746c9d5cec67b26ba8e7dcef4, ASSIGN in 520 msec 2023-08-06 09:53:45,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 in 1.0060 sec 2023-08-06 09:53:45,235 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] throttle.PressureAwareThroughputController(145): c8dd2ff746c9d5cec67b26ba8e7dcef4#cf2#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:45,235 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] throttle.PressureAwareThroughputController(145): c8dd2ff746c9d5cec67b26ba8e7dcef4#cf1#compaction#75 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:45,258 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 54ef08dd4b504efe8475195f380d1217: NONE, but ROW specified in column family configuration 2023-08-06 09:53:45,259 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/.tmp/cf2/54ef08dd4b504efe8475195f380d1217 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/54ef08dd4b504efe8475195f380d1217 2023-08-06 09:53:45,260 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 987c48d23a634634b5cf659c5556cd43: NONE, but ROW specified in column family configuration 2023-08-06 09:53:45,261 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/.tmp/cf1/987c48d23a634634b5cf659c5556cd43 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/987c48d23a634634b5cf659c5556cd43 2023-08-06 09:53:45,263 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 54ef08dd4b504efe8475195f380d1217: NONE, but ROW specified in column family configuration 2023-08-06 09:53:45,264 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2 of c8dd2ff746c9d5cec67b26ba8e7dcef4 into 54ef08dd4b504efe8475195f380d1217(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:45,264 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for c8dd2ff746c9d5cec67b26ba8e7dcef4: 2023-08-06 09:53:45,264 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4., storeName=c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2, priority=9, startTime=1691315625215; duration=0sec 2023-08-06 09:53:45,264 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:45,265 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 987c48d23a634634b5cf659c5556cd43: NONE, but ROW specified in column family configuration 2023-08-06 09:53:45,266 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1 of c8dd2ff746c9d5cec67b26ba8e7dcef4 into 987c48d23a634634b5cf659c5556cd43(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:45,266 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c8dd2ff746c9d5cec67b26ba8e7dcef4: 2023-08-06 09:53:45,266 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4., storeName=c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1, priority=9, startTime=1691315625214; duration=0sec 2023-08-06 09:53:45,266 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:45,480 DEBUG [Listener at localhost/36789] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-06 09:53:45,483 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49988, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-06 09:53:45,483 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40847] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,484 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40847] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-06 09:53:45,484 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40847] regionserver.HStore(1912): b6ad8f165ee195bcbbe04c83a68f9d95/cf1 is initiating major compaction (all files) 2023-08-06 09:53:45,484 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40847] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1c45621f; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:45,484 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b6ad8f165ee195bcbbe04c83a68f9d95/cf1 in testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,484 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40847] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-06 09:53:45,484 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/2cbf6f0f1c7249fb8285dc29516b8b1b] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/.tmp, totalSize=6.8 K 2023-08-06 09:53:45,485 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40847] regionserver.HStore(1912): b6ad8f165ee195bcbbe04c83a68f9d95/cf2 is initiating major compaction (all files) 2023-08-06 09:53:45,485 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40847] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6fe11d9c; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:45,485 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2259): Starting compaction of b6ad8f165ee195bcbbe04c83a68f9d95/cf2 in testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,485 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/feb6a81fb81b4480936aaedfe8fab6bf] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/.tmp, totalSize=6.8 K 2023-08-06 09:53:45,485 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 2cbf6f0f1c7249fb8285dc29516b8b1b, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691315622554 2023-08-06 09:53:45,485 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting feb6a81fb81b4480936aaedfe8fab6bf, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691315622554 2023-08-06 09:53:45,486 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40847] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,487 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40847] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-06 09:53:45,487 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40847] regionserver.HStore(1912): c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1 is initiating major compaction (all files) 2023-08-06 09:53:45,487 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40847] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@434a475f; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-06 09:53:45,487 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40847] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-06 09:53:45,487 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40847] regionserver.HStore(1912): c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2 is initiating major compaction (all files) 2023-08-06 09:53:45,487 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40847] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@34b2ea46; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-08-06 09:53:45,488 INFO [Listener at localhost/36789] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-08-06 09:53:45,490 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] throttle.PressureAwareThroughputController(145): b6ad8f165ee195bcbbe04c83a68f9d95#cf2#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:45,490 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] throttle.PressureAwareThroughputController(145): b6ad8f165ee195bcbbe04c83a68f9d95#cf1#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:45,501 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40847] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. is not mergeable because it has references 2023-08-06 09:53:45,502 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40847] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. is not mergeable because it has references 2023-08-06 09:53:45,507 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/.tmp/cf2/8ea1c38788c14d518b6d88a9f9085eb2 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8ea1c38788c14d518b6d88a9f9085eb2 2023-08-06 09:53:45,509 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/.tmp/cf1/f09e0298086a421d877b941f8391e80f as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/f09e0298086a421d877b941f8391e80f 2023-08-06 09:53:45,512 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in b6ad8f165ee195bcbbe04c83a68f9d95/cf2 of b6ad8f165ee195bcbbe04c83a68f9d95 into 8ea1c38788c14d518b6d88a9f9085eb2(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:45,512 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for b6ad8f165ee195bcbbe04c83a68f9d95: 2023-08-06 09:53:45,514 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in b6ad8f165ee195bcbbe04c83a68f9d95/cf1 of b6ad8f165ee195bcbbe04c83a68f9d95 into f09e0298086a421d877b941f8391e80f(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:45,514 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b6ad8f165ee195bcbbe04c83a68f9d95: 2023-08-06 09:53:45,514 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1691315625485; duration=0sec 2023-08-06 09:53:45,514 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1691315625484; duration=0sec 2023-08-06 09:53:45,514 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-08-06 09:53:45,514 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-08-06 09:53:45,514 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2259): Starting compaction of c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,514 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,514 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/54ef08dd4b504efe8475195f380d1217] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/.tmp, totalSize=5.0 K 2023-08-06 09:53:45,514 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/987c48d23a634634b5cf659c5556cd43] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/.tmp, totalSize=5.0 K 2023-08-06 09:53:45,514 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 54ef08dd4b504efe8475195f380d1217, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-08-06 09:53:45,514 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 987c48d23a634634b5cf659c5556cd43, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-08-06 09:53:45,517 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] throttle.PressureAwareThroughputController(145): c8dd2ff746c9d5cec67b26ba8e7dcef4#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:45,517 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] throttle.PressureAwareThroughputController(145): c8dd2ff746c9d5cec67b26ba8e7dcef4#cf1#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:53:45,530 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 83ea2d0658474daeb3fa29e5346d24de: NONE, but ROW specified in column family configuration 2023-08-06 09:53:45,530 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 466b871fae614fde8943fabf18e89b16: NONE, but ROW specified in column family configuration 2023-08-06 09:53:45,531 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/.tmp/cf1/83ea2d0658474daeb3fa29e5346d24de as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/83ea2d0658474daeb3fa29e5346d24de 2023-08-06 09:53:45,531 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/.tmp/cf2/466b871fae614fde8943fabf18e89b16 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/466b871fae614fde8943fabf18e89b16 2023-08-06 09:53:45,535 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 83ea2d0658474daeb3fa29e5346d24de: NONE, but ROW specified in column family configuration 2023-08-06 09:53:45,535 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 466b871fae614fde8943fabf18e89b16: NONE, but ROW specified in column family configuration 2023-08-06 09:53:45,536 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1 of c8dd2ff746c9d5cec67b26ba8e7dcef4 into 83ea2d0658474daeb3fa29e5346d24de(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:45,536 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c8dd2ff746c9d5cec67b26ba8e7dcef4: 2023-08-06 09:53:45,536 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1691315625487; duration=0sec 2023-08-06 09:53:45,536 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:45,536 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2 of c8dd2ff746c9d5cec67b26ba8e7dcef4 into 466b871fae614fde8943fabf18e89b16(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:53:45,536 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for c8dd2ff746c9d5cec67b26ba8e7dcef4: 2023-08-06 09:53:45,536 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1691315625487; duration=0sec 2023-08-06 09:53:45,536 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:53:45,610 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40847] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. is not mergeable because it has references 2023-08-06 09:53:45,610 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40847] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. is not mergeable because it has references 2023-08-06 09:53:45,616 DEBUG [Listener at localhost/36789] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-08-06 09:53:45,618 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:45,619 INFO [Listener at localhost/36789] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:45,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:45,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:45,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-08-06 09:53:45,623 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315625623"}]},"ts":"1691315625623"} 2023-08-06 09:53:45,624 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-08-06 09:53:45,626 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-08-06 09:53:45,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b6ad8f165ee195bcbbe04c83a68f9d95, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c8dd2ff746c9d5cec67b26ba8e7dcef4, UNASSIGN}] 2023-08-06 09:53:45,628 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c8dd2ff746c9d5cec67b26ba8e7dcef4, UNASSIGN 2023-08-06 09:53:45,628 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=c8dd2ff746c9d5cec67b26ba8e7dcef4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:45,628 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691315625628"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315625628"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315625628"}]},"ts":"1691315625628"} 2023-08-06 09:53:45,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure c8dd2ff746c9d5cec67b26ba8e7dcef4, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:53:45,631 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b6ad8f165ee195bcbbe04c83a68f9d95, UNASSIGN 2023-08-06 09:53:45,632 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=b6ad8f165ee195bcbbe04c83a68f9d95, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:45,632 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691315625632"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315625632"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315625632"}]},"ts":"1691315625632"} 2023-08-06 09:53:45,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure b6ad8f165ee195bcbbe04c83a68f9d95, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:53:45,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-08-06 09:53:45,781 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,781 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b6ad8f165ee195bcbbe04c83a68f9d95, disabling compactions & flushes 2023-08-06 09:53:45,781 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,781 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,781 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. after waiting 0 ms 2023-08-06 09:53:45,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,782 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-9e3315554c2b431cb6e517f18b1126e1, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-b08b61c6bf054db49e18d1dde5b4c9d4, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-0c5041247b1a4445a2d9a09dad5a2bf6, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/2cbf6f0f1c7249fb8285dc29516b8b1b, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f-bottom] to archive 2023-08-06 09:53:45,783 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:45,785 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,786 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,787 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,788 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-9e3315554c2b431cb6e517f18b1126e1 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-9e3315554c2b431cb6e517f18b1126e1 2023-08-06 09:53:45,789 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-b08b61c6bf054db49e18d1dde5b4c9d4 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-b08b61c6bf054db49e18d1dde5b4c9d4 2023-08-06 09:53:45,790 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-0c5041247b1a4445a2d9a09dad5a2bf6 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-0c5041247b1a4445a2d9a09dad5a2bf6 2023-08-06 09:53:45,791 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/2cbf6f0f1c7249fb8285dc29516b8b1b to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/2cbf6f0f1c7249fb8285dc29516b8b1b 2023-08-06 09:53:45,792 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,794 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-926e6638580649ca9fc04110695650d7, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-8eb1fe5ca9e14e9a87980bd73ffa5f42, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ad4354de17e04428affa1d0fb5b7716d, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/feb6a81fb81b4480936aaedfe8fab6bf, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2-bottom] to archive 2023-08-06 09:53:45,795 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:45,796 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,797 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,798 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,799 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-926e6638580649ca9fc04110695650d7 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-926e6638580649ca9fc04110695650d7 2023-08-06 09:53:45,800 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-8eb1fe5ca9e14e9a87980bd73ffa5f42 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-8eb1fe5ca9e14e9a87980bd73ffa5f42 2023-08-06 09:53:45,801 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ad4354de17e04428affa1d0fb5b7716d to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ad4354de17e04428affa1d0fb5b7716d 2023-08-06 09:53:45,802 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/feb6a81fb81b4480936aaedfe8fab6bf to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/feb6a81fb81b4480936aaedfe8fab6bf 2023-08-06 09:53:45,803 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,806 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-08-06 09:53:45,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:45,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95. 2023-08-06 09:53:45,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b6ad8f165ee195bcbbe04c83a68f9d95: 2023-08-06 09:53:45,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c8dd2ff746c9d5cec67b26ba8e7dcef4, disabling compactions & flushes 2023-08-06 09:53:45,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. after waiting 0 ms 2023-08-06 09:53:45,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,809 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=b6ad8f165ee195bcbbe04c83a68f9d95, regionState=CLOSED 2023-08-06 09:53:45,809 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691315625809"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315625809"}]},"ts":"1691315625809"} 2023-08-06 09:53:45,810 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-790ef4f3b74d4703bad0afb472b04e3d, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-edb3aa3b3c7e4f92b389cef33c59412a, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/987c48d23a634634b5cf659c5556cd43, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-dcc3df610e7a494f913b769400d29bde] to archive 2023-08-06 09:53:45,810 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:45,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-08-06 09:53:45,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure b6ad8f165ee195bcbbe04c83a68f9d95, server=jenkins-hbase4.apache.org,40847,1691315596668 in 177 msec 2023-08-06 09:53:45,812 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b6ad8f165ee195bcbbe04c83a68f9d95, UNASSIGN in 185 msec 2023-08-06 09:53:45,813 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/38755fc40e994630994caadd509e1987.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,814 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/aac40dfa5c7e49c297c83bb015d4230b.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,815 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/0591566a9f6745aa986b3a9b086cfc2f.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,816 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-790ef4f3b74d4703bad0afb472b04e3d to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-790ef4f3b74d4703bad0afb472b04e3d 2023-08-06 09:53:45,817 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-edb3aa3b3c7e4f92b389cef33c59412a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-edb3aa3b3c7e4f92b389cef33c59412a 2023-08-06 09:53:45,818 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/987c48d23a634634b5cf659c5556cd43 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/987c48d23a634634b5cf659c5556cd43 2023-08-06 09:53:45,819 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-dcc3df610e7a494f913b769400d29bde to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-dcc3df610e7a494f913b769400d29bde 2023-08-06 09:53:45,821 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-65789a4ef0c54644b57e08ef639bfb7a, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-315e517a144c4e82a30f12f8c52bc7d3, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/54ef08dd4b504efe8475195f380d1217, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ecbcccb9f84842c08bc61a62bfe7751e] to archive 2023-08-06 09:53:45,821 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:53:45,822 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/8f0616390d844cb296df3bb4538143ae.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,823 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/6b48656f275e4615a2166199f56512d6.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,824 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/89f0026456a347e0b82ea6351430d9a6.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,825 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/8f623aeded074395a08d1a701de7f4e2.c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,826 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-65789a4ef0c54644b57e08ef639bfb7a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-65789a4ef0c54644b57e08ef639bfb7a 2023-08-06 09:53:45,827 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-315e517a144c4e82a30f12f8c52bc7d3 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-315e517a144c4e82a30f12f8c52bc7d3 2023-08-06 09:53:45,828 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/54ef08dd4b504efe8475195f380d1217 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/54ef08dd4b504efe8475195f380d1217 2023-08-06 09:53:45,829 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ecbcccb9f84842c08bc61a62bfe7751e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/testSplitTableRegionDeletedRowsDaughter=c7bf488ae570a703d8494b8ebc889fae-ecbcccb9f84842c08bc61a62bfe7751e 2023-08-06 09:53:45,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-08-06 09:53:45,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:45,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4. 2023-08-06 09:53:45,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c8dd2ff746c9d5cec67b26ba8e7dcef4: 2023-08-06 09:53:45,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,835 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=c8dd2ff746c9d5cec67b26ba8e7dcef4, regionState=CLOSED 2023-08-06 09:53:45,835 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1691315625835"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315625835"}]},"ts":"1691315625835"} 2023-08-06 09:53:45,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-08-06 09:53:45,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure c8dd2ff746c9d5cec67b26ba8e7dcef4, server=jenkins-hbase4.apache.org,40847,1691315596668 in 208 msec 2023-08-06 09:53:45,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-08-06 09:53:45,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c8dd2ff746c9d5cec67b26ba8e7dcef4, UNASSIGN in 211 msec 2023-08-06 09:53:45,840 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315625840"}]},"ts":"1691315625840"} 2023-08-06 09:53:45,841 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-08-06 09:53:45,842 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-08-06 09:53:45,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 223 msec 2023-08-06 09:53:45,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-08-06 09:53:45,925 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-08-06 09:53:45,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:45,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:45,928 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:45,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-08-06 09:53:45,929 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:45,934 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,934 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,934 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,936 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/recovered.edits] 2023-08-06 09:53:45,936 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/recovered.edits] 2023-08-06 09:53:45,936 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/recovered.edits] 2023-08-06 09:53:45,939 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/83ea2d0658474daeb3fa29e5346d24de to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf1/83ea2d0658474daeb3fa29e5346d24de 2023-08-06 09:53:45,939 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/f09e0298086a421d877b941f8391e80f to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf1/f09e0298086a421d877b941f8391e80f 2023-08-06 09:53:45,939 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0591566a9f6745aa986b3a9b086cfc2f 2023-08-06 09:53:45,941 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0c5041247b1a4445a2d9a09dad5a2bf6 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/0c5041247b1a4445a2d9a09dad5a2bf6 2023-08-06 09:53:45,942 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/466b871fae614fde8943fabf18e89b16 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/cf2/466b871fae614fde8943fabf18e89b16 2023-08-06 09:53:45,942 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8ea1c38788c14d518b6d88a9f9085eb2 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/cf2/8ea1c38788c14d518b6d88a9f9085eb2 2023-08-06 09:53:45,942 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/38755fc40e994630994caadd509e1987 2023-08-06 09:53:45,944 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/588cf2c6be364c6aaa51fdca5d7f5cf6 2023-08-06 09:53:45,945 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/recovered.edits/113.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4/recovered.edits/113.seqid 2023-08-06 09:53:45,945 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/790ef4f3b74d4703bad0afb472b04e3d to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/790ef4f3b74d4703bad0afb472b04e3d 2023-08-06 09:53:45,945 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/recovered.edits/113.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95/recovered.edits/113.seqid 2023-08-06 09:53:45,945 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:53:45,946 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b6ad8f165ee195bcbbe04c83a68f9d95 2023-08-06 09:53:45,947 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/9e3315554c2b431cb6e517f18b1126e1 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/9e3315554c2b431cb6e517f18b1126e1 2023-08-06 09:53:45,948 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/aac40dfa5c7e49c297c83bb015d4230b 2023-08-06 09:53:45,949 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/b08b61c6bf054db49e18d1dde5b4c9d4 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/b08b61c6bf054db49e18d1dde5b4c9d4 2023-08-06 09:53:45,951 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/dcc3df610e7a494f913b769400d29bde to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/dcc3df610e7a494f913b769400d29bde 2023-08-06 09:53:45,952 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/edb3aa3b3c7e4f92b389cef33c59412a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf1/edb3aa3b3c7e4f92b389cef33c59412a 2023-08-06 09:53:45,955 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/315e517a144c4e82a30f12f8c52bc7d3 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/315e517a144c4e82a30f12f8c52bc7d3 2023-08-06 09:53:45,956 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/65789a4ef0c54644b57e08ef639bfb7a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/65789a4ef0c54644b57e08ef639bfb7a 2023-08-06 09:53:45,957 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/6b48656f275e4615a2166199f56512d6 2023-08-06 09:53:45,959 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/89f0026456a347e0b82ea6351430d9a6 2023-08-06 09:53:45,960 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8eb1fe5ca9e14e9a87980bd73ffa5f42 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8eb1fe5ca9e14e9a87980bd73ffa5f42 2023-08-06 09:53:45,961 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f0616390d844cb296df3bb4538143ae 2023-08-06 09:53:45,962 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/8f623aeded074395a08d1a701de7f4e2 2023-08-06 09:53:45,963 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/926e6638580649ca9fc04110695650d7 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/926e6638580649ca9fc04110695650d7 2023-08-06 09:53:45,964 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ad4354de17e04428affa1d0fb5b7716d to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ad4354de17e04428affa1d0fb5b7716d 2023-08-06 09:53:45,966 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ecbcccb9f84842c08bc61a62bfe7751e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/cf2/ecbcccb9f84842c08bc61a62bfe7751e 2023-08-06 09:53:45,968 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/recovered.edits/106.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae/recovered.edits/106.seqid 2023-08-06 09:53:45,968 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c7bf488ae570a703d8494b8ebc889fae 2023-08-06 09:53:45,968 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-08-06 09:53:45,971 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:45,973 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-08-06 09:53:45,974 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-08-06 09:53:45,975 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:45,976 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-08-06 09:53:45,976 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315625976"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:45,976 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315625976"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:45,976 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315625976"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:45,978 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-06 09:53:45,978 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c7bf488ae570a703d8494b8ebc889fae, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1691315622047.c7bf488ae570a703d8494b8ebc889fae.', STARTKEY => '', ENDKEY => ''}, {ENCODED => b6ad8f165ee195bcbbe04c83a68f9d95, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1691315624218.b6ad8f165ee195bcbbe04c83a68f9d95.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => c8dd2ff746c9d5cec67b26ba8e7dcef4, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1691315624218.c8dd2ff746c9d5cec67b26ba8e7dcef4.', STARTKEY => '60', ENDKEY => ''}] 2023-08-06 09:53:45,978 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-08-06 09:53:45,978 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691315625978"}]},"ts":"9223372036854775807"} 2023-08-06 09:53:45,979 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-08-06 09:53:45,981 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:53:45,982 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 55 msec 2023-08-06 09:53:46,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-08-06 09:53:46,031 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-08-06 09:53:46,046 INFO [Listener at localhost/36789] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=413 (was 402) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:37340 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:37320 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:37708 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:37740 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:59052 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:37814 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:37754 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:45780 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:40744 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:40742 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:37730 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:40758 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:40702 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:40847-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:37720 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:45464 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45a8b49f-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-83447583_17 at /127.0.0.1:40718 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=853 (was 862), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=236 (was 248), ProcessCount=175 (was 175), AvailableMemoryMB=2335 (was 2346) 2023-08-06 09:53:46,060 INFO [Listener at localhost/36789] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=413, OpenFileDescriptor=853, MaxFileDescriptor=60000, SystemLoadAverage=236, ProcessCount=175, AvailableMemoryMB=2334 2023-08-06 09:53:46,060 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:53:46,062 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-06 09:53:46,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-06 09:53:46,063 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:53:46,065 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-08-06 09:53:46,066 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-06 09:53:46,067 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:46,067 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4 empty. 2023-08-06 09:53:46,068 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:46,068 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-08-06 09:53:46,082 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-08-06 09:53:46,083 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => 74bda565887a7bc02c6c0743649997a4, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp 2023-08-06 09:53:46,094 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:46,094 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing 74bda565887a7bc02c6c0743649997a4, disabling compactions & flushes 2023-08-06 09:53:46,094 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:46,094 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:46,094 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. after waiting 0 ms 2023-08-06 09:53:46,094 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:46,094 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:46,094 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for 74bda565887a7bc02c6c0743649997a4: 2023-08-06 09:53:46,096 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-08-06 09:53:46,097 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315626097"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315626097"}]},"ts":"1691315626097"} 2023-08-06 09:53:46,098 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-06 09:53:46,099 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-06 09:53:46,099 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315626099"}]},"ts":"1691315626099"} 2023-08-06 09:53:46,100 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-08-06 09:53:46,104 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:53:46,105 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:53:46,105 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:53:46,105 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:53:46,105 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:53:46,105 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74bda565887a7bc02c6c0743649997a4, ASSIGN}] 2023-08-06 09:53:46,106 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74bda565887a7bc02c6c0743649997a4, ASSIGN 2023-08-06 09:53:46,107 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74bda565887a7bc02c6c0743649997a4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39585,1691315596578; forceNewPlan=false, retain=false 2023-08-06 09:53:46,257 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:53:46,260 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=74bda565887a7bc02c6c0743649997a4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:46,260 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315626260"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315626260"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315626260"}]},"ts":"1691315626260"} 2023-08-06 09:53:46,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 74bda565887a7bc02c6c0743649997a4, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:53:46,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:46,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 74bda565887a7bc02c6c0743649997a4, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:53:46,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:46,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:46,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:46,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:46,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:46,416 INFO [StoreOpener-74bda565887a7bc02c6c0743649997a4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:46,417 DEBUG [StoreOpener-74bda565887a7bc02c6c0743649997a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/cf1 2023-08-06 09:53:46,417 DEBUG [StoreOpener-74bda565887a7bc02c6c0743649997a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/cf1 2023-08-06 09:53:46,418 INFO [StoreOpener-74bda565887a7bc02c6c0743649997a4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74bda565887a7bc02c6c0743649997a4 columnFamilyName cf1 2023-08-06 09:53:46,418 INFO [StoreOpener-74bda565887a7bc02c6c0743649997a4-1] regionserver.HStore(310): Store=74bda565887a7bc02c6c0743649997a4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:46,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:46,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:46,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:46,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:53:46,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 74bda565887a7bc02c6c0743649997a4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70665025, jitterRate=0.052990928292274475}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-06 09:53:46,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 74bda565887a7bc02c6c0743649997a4: 2023-08-06 09:53:46,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4., pid=87, masterSystemTime=1691315626412 2023-08-06 09:53:46,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:46,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:46,426 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=74bda565887a7bc02c6c0743649997a4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:46,426 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315626426"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315626426"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315626426"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315626426"}]},"ts":"1691315626426"} 2023-08-06 09:53:46,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-08-06 09:53:46,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 74bda565887a7bc02c6c0743649997a4, server=jenkins-hbase4.apache.org,39585,1691315596578 in 166 msec 2023-08-06 09:53:46,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-08-06 09:53:46,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74bda565887a7bc02c6c0743649997a4, ASSIGN in 324 msec 2023-08-06 09:53:46,431 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-06 09:53:46,431 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315626431"}]},"ts":"1691315626431"} 2023-08-06 09:53:46,432 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-08-06 09:53:46,438 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-08-06 09:53:46,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 375 msec 2023-08-06 09:53:46,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-06 09:53:46,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:53:46,596 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-08-06 09:53:46,597 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-08-06 09:53:46,597 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-08-06 09:53:46,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-08-06 09:53:46,616 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74bda565887a7bc02c6c0743649997a4, REOPEN/MOVE}] 2023-08-06 09:53:46,618 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74bda565887a7bc02c6c0743649997a4, REOPEN/MOVE 2023-08-06 09:53:46,618 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=74bda565887a7bc02c6c0743649997a4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:46,618 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315626618"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315626618"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315626618"}]},"ts":"1691315626618"} 2023-08-06 09:53:46,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 74bda565887a7bc02c6c0743649997a4, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:53:46,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:46,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 74bda565887a7bc02c6c0743649997a4, disabling compactions & flushes 2023-08-06 09:53:46,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:46,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:46,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. after waiting 0 ms 2023-08-06 09:53:46,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:46,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-08-06 09:53:46,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:53:46,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:46,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 74bda565887a7bc02c6c0743649997a4: 2023-08-06 09:53:46,779 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 74bda565887a7bc02c6c0743649997a4 to self. 2023-08-06 09:53:46,781 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:46,781 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=74bda565887a7bc02c6c0743649997a4, regionState=CLOSED 2023-08-06 09:53:46,781 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315626781"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315626781"}]},"ts":"1691315626781"} 2023-08-06 09:53:46,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-08-06 09:53:46,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 74bda565887a7bc02c6c0743649997a4, server=jenkins-hbase4.apache.org,39585,1691315596578 in 163 msec 2023-08-06 09:53:46,785 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74bda565887a7bc02c6c0743649997a4, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,39585,1691315596578; forceNewPlan=false, retain=true 2023-08-06 09:53:46,935 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:53:46,936 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=74bda565887a7bc02c6c0743649997a4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:46,936 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315626936"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315626936"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315626936"}]},"ts":"1691315626936"} 2023-08-06 09:53:46,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 74bda565887a7bc02c6c0743649997a4, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:53:47,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:47,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 74bda565887a7bc02c6c0743649997a4, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:53:47,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:47,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:47,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:47,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:47,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:47,094 INFO [StoreOpener-74bda565887a7bc02c6c0743649997a4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:47,095 DEBUG [StoreOpener-74bda565887a7bc02c6c0743649997a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/cf1 2023-08-06 09:53:47,095 DEBUG [StoreOpener-74bda565887a7bc02c6c0743649997a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/cf1 2023-08-06 09:53:47,096 INFO [StoreOpener-74bda565887a7bc02c6c0743649997a4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74bda565887a7bc02c6c0743649997a4 columnFamilyName cf1 2023-08-06 09:53:47,096 INFO [StoreOpener-74bda565887a7bc02c6c0743649997a4-1] regionserver.HStore(310): Store=74bda565887a7bc02c6c0743649997a4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:53:47,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:47,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:47,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:53:47,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 74bda565887a7bc02c6c0743649997a4; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69572635, jitterRate=0.036713048815727234}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-06 09:53:47,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 74bda565887a7bc02c6c0743649997a4: 2023-08-06 09:53:47,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4., pid=92, masterSystemTime=1691315627090 2023-08-06 09:53:47,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:47,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:53:47,103 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=74bda565887a7bc02c6c0743649997a4, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:53:47,103 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315627103"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315627103"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315627103"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315627103"}]},"ts":"1691315627103"} 2023-08-06 09:53:47,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-08-06 09:53:47,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 74bda565887a7bc02c6c0743649997a4, server=jenkins-hbase4.apache.org,39585,1691315596578 in 166 msec 2023-08-06 09:53:47,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-08-06 09:53:47,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74bda565887a7bc02c6c0743649997a4, REOPEN/MOVE in 490 msec 2023-08-06 09:53:47,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-08-06 09:53:47,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 509 msec 2023-08-06 09:53:47,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d, ASSIGN}] 2023-08-06 09:53:47,114 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d, ASSIGN 2023-08-06 09:53:47,114 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-06 09:53:47,264 DEBUG [jenkins-hbase4:33207] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:53:47,266 DEBUG [jenkins-hbase4:33207] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:53:47,266 DEBUG [jenkins-hbase4:33207] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:53:47,266 DEBUG [jenkins-hbase4:33207] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:53:47,266 DEBUG [jenkins-hbase4:33207] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-08-06 09:53:47,268 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=070e4b344b2814c926ad7094797c209d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:53:47,268 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315627268"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1691315627268"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1691315627268"}]},"ts":"1691315627268"} 2023-08-06 09:53:47,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 070e4b344b2814c926ad7094797c209d, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:53:47,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d. 2023-08-06 09:53:47,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 070e4b344b2814c926ad7094797c209d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-08-06 09:53:47,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:53:47,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 070e4b344b2814c926ad7094797c209d 2023-08-06 09:53:47,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:53:47,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 070e4b344b2814c926ad7094797c209d 2023-08-06 09:53:47,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 070e4b344b2814c926ad7094797c209d 2023-08-06 09:53:47,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:47,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:47,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:47,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:47,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:47,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:48,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:48,075 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-08-06 09:53:48,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:48,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:48,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:48,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:48,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:48,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:48,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:48,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:48,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:49,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:49,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:49,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:49,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:49,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:49,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:49,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:49,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:49,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:49,923 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-06 09:53:49,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:50,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:50,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:50,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:50,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:50,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:50,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:50,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:50,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:50,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:50,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:51,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:51,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:51,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:51,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:51,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:51,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:51,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:51,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:51,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:51,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:52,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:52,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:52,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:52,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:52,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:52,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:52,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:52,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:52,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:52,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:53,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:53,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:53,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:53,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:53,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:53,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:53,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:53,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:53,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:53,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:54,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:54,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:54,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:54,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:54,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:54,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:54,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:54,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:54,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:54,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:55,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:55,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:55,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:55,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:55,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:55,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:55,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:55,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:55,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:55,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:56,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:56,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:56,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:56,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:56,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:56,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:56,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:56,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:56,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:56,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:57,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:57,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:57,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:57,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:57,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:57,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:57,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:57,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:57,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:57,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:58,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:58,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:58,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:58,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:58,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:58,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:58,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:58,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:58,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:58,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:59,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:59,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:59,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:59,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:59,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:59,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:59,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:59,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:59,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:53:59,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:00,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:00,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:00,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:00,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:00,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:00,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:00,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:00,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:00,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:00,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:01,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:01,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:01,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:01,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:01,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:01,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:01,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:01,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:01,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:01,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:02,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:02,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:02,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:02,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:02,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:02,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:02,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:02,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:02,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:02,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:03,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:03,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:03,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:03,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:03,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:03,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:03,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:03,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:03,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:03,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:04,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:04,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:04,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:04,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:04,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:04,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:04,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:04,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:04,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:04,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:05,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:05,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:05,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:05,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:05,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:05,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:05,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:05,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:05,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:05,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:06,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:06,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:06,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:06,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:06,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:06,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:06,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:06,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:06,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:06,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:07,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:07,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:07,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:07,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:07,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:07,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:07,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:07,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:07,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:07,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:08,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:08,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:08,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:08,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:08,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:08,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:08,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:08,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:08,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:08,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:09,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:09,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:09,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:09,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:09,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:09,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:09,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:09,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:09,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:09,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:10,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:10,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:10,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:10,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:10,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:10,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:10,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:10,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:10,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:10,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:11,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:11,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:11,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:11,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:11,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:11,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:11,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:11,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:11,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:11,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:12,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:12,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:12,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:12,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:12,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:12,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:12,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:12,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:12,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:12,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:13,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:13,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:13,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:13,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:13,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:13,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:13,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:13,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:13,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:13,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:14,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:14,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:14,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:14,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:14,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:14,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:14,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:14,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:14,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:14,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:15,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:15,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:15,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:15,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:15,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:15,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:15,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:15,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:15,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:15,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:16,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:16,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:16,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:16,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:16,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:16,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:16,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:16,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:16,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:16,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:17,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:17,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:17,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:17,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:17,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:17,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:17,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:17,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:17,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:17,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:18,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:18,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:18,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:18,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:18,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:18,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:18,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:18,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:18,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:18,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:19,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:19,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:19,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:19,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:19,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:19,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:19,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:19,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:19,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:19,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:20,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:20,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:20,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:20,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:20,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:20,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:20,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:20,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:20,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:20,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:21,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:21,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:21,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:21,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:21,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:21,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:21,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:21,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:21,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:21,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:22,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:22,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:22,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:22,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:22,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:22,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:22,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:22,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:22,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:23,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:23,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:23,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:23,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:23,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:23,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:23,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:23,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:23,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:23,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:24,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:24,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:24,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:24,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:24,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:24,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:24,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:24,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:24,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:24,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:25,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:25,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:25,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:25,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:25,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:25,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:25,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:25,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:25,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:25,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:26,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:26,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:26,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:26,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:26,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:26,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:26,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:26,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:26,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:26,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:27,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:27,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:27,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:27,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:27,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:27,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:27,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:27,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:27,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:27,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:28,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:28,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:28,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:28,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:28,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:28,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:28,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:28,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:28,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:28,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:29,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:29,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:29,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:29,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:29,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:29,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:29,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:29,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:29,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:29,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:30,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:30,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:30,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:30,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:30,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:30,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:30,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:30,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:30,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:30,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:31,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:31,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:31,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:31,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:31,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:31,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:31,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:31,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:31,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:31,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:32,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:32,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:32,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:32,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:32,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:32,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:32,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:32,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:32,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:32,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:33,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:33,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:33,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:33,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:33,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:33,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:33,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:33,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:33,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:33,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:34,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:34,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:34,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:34,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:34,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:34,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:34,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:34,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:34,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:34,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:35,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:35,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:35,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:35,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:35,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:35,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:35,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:35,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:35,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:35,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:36,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:36,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:36,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:36,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:36,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:36,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:36,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:36,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:36,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:36,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:37,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:37,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:37,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:37,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:37,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:37,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:37,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:37,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:37,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:37,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:38,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:38,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:38,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:38,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:38,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:38,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:38,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:38,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:38,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:38,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:39,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:39,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:39,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:39,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:39,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:39,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:39,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:39,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:39,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:39,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:40,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:40,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:40,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:40,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:40,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:40,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:40,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:40,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:40,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:40,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:41,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:41,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:41,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:41,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:41,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:41,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:41,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:41,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:41,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:41,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:42,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:42,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:42,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:42,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:42,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:42,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:42,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:42,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:42,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:42,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:43,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:43,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:43,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:43,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:43,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:43,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:43,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:43,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:43,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:43,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:44,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:44,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:44,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:44,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:44,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:44,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:44,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:44,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:44,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:44,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:45,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:45,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:45,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:45,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:45,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:45,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:45,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:45,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:45,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:45,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:46,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:46,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:46,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:46,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:46,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:46,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:46,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:46,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-06 09:54:46,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:46,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-06 09:54:46,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:46,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:46,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-06 09:54:47,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:47,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:47,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:47,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-06 09:54:47,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:47,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:47,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:47,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:47,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:47,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-06 09:54:47,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:47,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:48,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:48,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:48,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:48,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:48,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:48,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:48,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:48,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:48,800 DEBUG [Listener at localhost/36789] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:48,801 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:48,802 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:48,802 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:48,802 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:48,804 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 74bda565887a7bc02c6c0743649997a4, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:48,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 exec-time=6 msec 2023-08-06 09:54:48,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:48,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-06 09:54:49,054 INFO [StoreOpener-070e4b344b2814c926ad7094797c209d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 070e4b344b2814c926ad7094797c209d 2023-08-06 09:54:49,057 DEBUG [StoreOpener-070e4b344b2814c926ad7094797c209d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/cf1 2023-08-06 09:54:49,057 DEBUG [StoreOpener-070e4b344b2814c926ad7094797c209d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/cf1 2023-08-06 09:54:49,058 INFO [StoreOpener-070e4b344b2814c926ad7094797c209d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 070e4b344b2814c926ad7094797c209d columnFamilyName cf1 2023-08-06 09:54:49,059 INFO [StoreOpener-070e4b344b2814c926ad7094797c209d-1] regionserver.HStore(310): Store=070e4b344b2814c926ad7094797c209d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:54:49,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 070e4b344b2814c926ad7094797c209d; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74022399, jitterRate=0.10301969945430756}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-06 09:54:49,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 070e4b344b2814c926ad7094797c209d: 2023-08-06 09:54:49,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d., pid=94, masterSystemTime=1691315627423 2023-08-06 09:54:49,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d. 2023-08-06 09:54:49,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d. 2023-08-06 09:54:49,062 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=070e4b344b2814c926ad7094797c209d, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:49,062 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315689062"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1691315689062"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1691315689062"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1691315689062"}]},"ts":"1691315689062"} 2023-08-06 09:54:49,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-08-06 09:54:49,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 070e4b344b2814c926ad7094797c209d, server=jenkins-hbase4.apache.org,40847,1691315596668 in 1 mins, 1.794 sec 2023-08-06 09:54:49,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-08-06 09:54:49,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d, ASSIGN in 1 mins, 1.953 sec 2023-08-06 09:54:49,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.49 sec 2023-08-06 09:54:49,069 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1691315626570","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62498,"client":"172.31.14.131:35758","queuetimems":0,"class":"HMaster"} 2023-08-06 09:54:49,069 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 607, skipping over this response of 3 bytes 2023-08-06 09:54:51,052 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up 2023-08-06 09:54:51,053 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:54:51,055 INFO [Listener at localhost/36789] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:51,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:51,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:51,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-08-06 09:54:51,059 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315691059"}]},"ts":"1691315691059"} 2023-08-06 09:54:51,060 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-08-06 09:54:51,062 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-08-06 09:54:51,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74bda565887a7bc02c6c0743649997a4, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d, UNASSIGN}] 2023-08-06 09:54:51,063 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d, UNASSIGN 2023-08-06 09:54:51,064 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=070e4b344b2814c926ad7094797c209d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:51,064 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315691063"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1691315691063"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1691315691063"}]},"ts":"1691315691063"} 2023-08-06 09:54:51,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 070e4b344b2814c926ad7094797c209d, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:54:51,067 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74bda565887a7bc02c6c0743649997a4, UNASSIGN 2023-08-06 09:54:51,068 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=74bda565887a7bc02c6c0743649997a4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:51,068 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315691068"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315691068"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315691068"}]},"ts":"1691315691068"} 2023-08-06 09:54:51,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 74bda565887a7bc02c6c0743649997a4, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:54:51,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-08-06 09:54:51,217 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 070e4b344b2814c926ad7094797c209d 2023-08-06 09:54:51,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 070e4b344b2814c926ad7094797c209d, disabling compactions & flushes 2023-08-06 09:54:51,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d. 2023-08-06 09:54:51,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d. 2023-08-06 09:54:51,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d. after waiting 0 ms 2023-08-06 09:54:51,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d. 2023-08-06 09:54:51,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:54:51,219 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d. 2023-08-06 09:54:51,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 070e4b344b2814c926ad7094797c209d: 2023-08-06 09:54:51,220 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:54:51,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 74bda565887a7bc02c6c0743649997a4, disabling compactions & flushes 2023-08-06 09:54:51,221 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:54:51,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:54:51,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. after waiting 0 ms 2023-08-06 09:54:51,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:54:51,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 070e4b344b2814c926ad7094797c209d 2023-08-06 09:54:51,222 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=070e4b344b2814c926ad7094797c209d, regionState=CLOSED 2023-08-06 09:54:51,223 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315691222"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1691315691222"}]},"ts":"1691315691222"} 2023-08-06 09:54:51,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-08-06 09:54:51,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-08-06 09:54:51,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 070e4b344b2814c926ad7094797c209d, server=jenkins-hbase4.apache.org,40847,1691315596668 in 159 msec 2023-08-06 09:54:51,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:54:51,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4. 2023-08-06 09:54:51,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 74bda565887a7bc02c6c0743649997a4: 2023-08-06 09:54:51,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d, UNASSIGN in 169 msec 2023-08-06 09:54:51,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 74bda565887a7bc02c6c0743649997a4 2023-08-06 09:54:51,233 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=74bda565887a7bc02c6c0743649997a4, regionState=CLOSED 2023-08-06 09:54:51,234 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1691315691233"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315691233"}]},"ts":"1691315691233"} 2023-08-06 09:54:51,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-08-06 09:54:51,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 74bda565887a7bc02c6c0743649997a4, server=jenkins-hbase4.apache.org,39585,1691315596578 in 166 msec 2023-08-06 09:54:51,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-08-06 09:54:51,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74bda565887a7bc02c6c0743649997a4, UNASSIGN in 174 msec 2023-08-06 09:54:51,238 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315691238"}]},"ts":"1691315691238"} 2023-08-06 09:54:51,239 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-08-06 09:54:51,241 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-08-06 09:54:51,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 185 msec 2023-08-06 09:54:51,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-08-06 09:54:51,361 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-08-06 09:54:51,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:51,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:51,364 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:51,365 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:51,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-08-06 09:54:51,369 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4 2023-08-06 09:54:51,371 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/recovered.edits] 2023-08-06 09:54:51,375 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/recovered.edits/7.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4/recovered.edits/7.seqid 2023-08-06 09:54:51,375 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74bda565887a7bc02c6c0743649997a4 2023-08-06 09:54:51,376 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-08-06 09:54:51,378 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:51,380 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-08-06 09:54:51,382 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-08-06 09:54:51,383 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:51,383 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-08-06 09:54:51,383 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315691383"}]},"ts":"9223372036854775807"} 2023-08-06 09:54:51,383 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315691383"}]},"ts":"9223372036854775807"} 2023-08-06 09:54:51,385 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-08-06 09:54:51,385 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 74bda565887a7bc02c6c0743649997a4, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691315626063.74bda565887a7bc02c6c0743649997a4.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 070e4b344b2814c926ad7094797c209d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1691315626063_0001.070e4b344b2814c926ad7094797c209d.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-08-06 09:54:51,385 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-08-06 09:54:51,385 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691315691385"}]},"ts":"9223372036854775807"} 2023-08-06 09:54:51,386 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-08-06 09:54:51,388 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:51,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 26 msec 2023-08-06 09:54:51,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-08-06 09:54:51,467 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-08-06 09:54:51,480 INFO [Listener at localhost/36789] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=348 (was 413), OpenFileDescriptor=658 (was 853), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=146 (was 236), ProcessCount=171 (was 175), AvailableMemoryMB=2321 (was 2334) 2023-08-06 09:54:51,492 INFO [Listener at localhost/36789] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=349, OpenFileDescriptor=658, MaxFileDescriptor=60000, SystemLoadAverage=146, ProcessCount=171, AvailableMemoryMB=2321 2023-08-06 09:54:51,492 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:54:51,495 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-06 09:54:51,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-06 09:54:51,496 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:51,497 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-08-06 09:54:51,498 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-06 09:54:51,499 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:51,500 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89 empty. 2023-08-06 09:54:51,500 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:51,500 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-08-06 09:54:51,510 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-08-06 09:54:51,511 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => 88b081660e8b4c6ad85dd3e66e468f89, NAME => 'testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp 2023-08-06 09:54:51,518 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:54:51,518 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing 88b081660e8b4c6ad85dd3e66e468f89, disabling compactions & flushes 2023-08-06 09:54:51,518 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:51,518 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:51,518 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. after waiting 0 ms 2023-08-06 09:54:51,518 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:51,518 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:51,518 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for 88b081660e8b4c6ad85dd3e66e468f89: 2023-08-06 09:54:51,520 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-08-06 09:54:51,521 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1691315691521"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315691521"}]},"ts":"1691315691521"} 2023-08-06 09:54:51,522 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-06 09:54:51,523 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-06 09:54:51,523 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315691523"}]},"ts":"1691315691523"} 2023-08-06 09:54:51,524 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-08-06 09:54:51,527 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:54:51,528 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:54:51,528 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:54:51,528 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:54:51,528 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:54:51,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=88b081660e8b4c6ad85dd3e66e468f89, ASSIGN}] 2023-08-06 09:54:51,529 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=88b081660e8b4c6ad85dd3e66e468f89, ASSIGN 2023-08-06 09:54:51,530 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=88b081660e8b4c6ad85dd3e66e468f89, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39585,1691315596578; forceNewPlan=false, retain=false 2023-08-06 09:54:51,680 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:54:51,681 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=88b081660e8b4c6ad85dd3e66e468f89, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:51,681 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1691315691681"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315691681"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315691681"}]},"ts":"1691315691681"} 2023-08-06 09:54:51,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 88b081660e8b4c6ad85dd3e66e468f89, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:54:51,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:51,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 88b081660e8b4c6ad85dd3e66e468f89, NAME => 'testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:54:51,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:54:51,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:51,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:54:51,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:51,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:51,839 INFO [StoreOpener-88b081660e8b4c6ad85dd3e66e468f89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:51,840 DEBUG [StoreOpener-88b081660e8b4c6ad85dd3e66e468f89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1 2023-08-06 09:54:51,840 DEBUG [StoreOpener-88b081660e8b4c6ad85dd3e66e468f89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1 2023-08-06 09:54:51,841 INFO [StoreOpener-88b081660e8b4c6ad85dd3e66e468f89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88b081660e8b4c6ad85dd3e66e468f89 columnFamilyName cf1 2023-08-06 09:54:51,841 INFO [StoreOpener-88b081660e8b4c6ad85dd3e66e468f89-1] regionserver.HStore(310): Store=88b081660e8b4c6ad85dd3e66e468f89/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:54:51,841 INFO [StoreOpener-88b081660e8b4c6ad85dd3e66e468f89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:51,843 DEBUG [StoreOpener-88b081660e8b4c6ad85dd3e66e468f89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2 2023-08-06 09:54:51,843 DEBUG [StoreOpener-88b081660e8b4c6ad85dd3e66e468f89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2 2023-08-06 09:54:51,843 INFO [StoreOpener-88b081660e8b4c6ad85dd3e66e468f89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88b081660e8b4c6ad85dd3e66e468f89 columnFamilyName cf2 2023-08-06 09:54:51,844 INFO [StoreOpener-88b081660e8b4c6ad85dd3e66e468f89-1] regionserver.HStore(310): Store=88b081660e8b4c6ad85dd3e66e468f89/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:54:51,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:51,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:51,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:54:51,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:51,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:54:51,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 88b081660e8b4c6ad85dd3e66e468f89; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66114222, jitterRate=-0.01482132077217102}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:54:51,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 88b081660e8b4c6ad85dd3e66e468f89: 2023-08-06 09:54:51,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89., pid=104, masterSystemTime=1691315691834 2023-08-06 09:54:51,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:51,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:51,853 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=88b081660e8b4c6ad85dd3e66e468f89, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:51,853 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1691315691853"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315691853"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315691853"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315691853"}]},"ts":"1691315691853"} 2023-08-06 09:54:51,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-08-06 09:54:51,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 88b081660e8b4c6ad85dd3e66e468f89, server=jenkins-hbase4.apache.org,39585,1691315596578 in 172 msec 2023-08-06 09:54:51,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-08-06 09:54:51,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=88b081660e8b4c6ad85dd3e66e468f89, ASSIGN in 328 msec 2023-08-06 09:54:51,859 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-06 09:54:51,859 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315691859"}]},"ts":"1691315691859"} 2023-08-06 09:54:51,860 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-08-06 09:54:51,863 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-08-06 09:54:51,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 368 msec 2023-08-06 09:54:52,004 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:52,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-06 09:54:52,005 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-06 09:54:52,005 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,005 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-06 09:54:52,005 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-06 09:54:52,006 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,006 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,007 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,007 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,007 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:52,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,009 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,009 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,009 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,009 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,009 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,009 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,009 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,009 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,009 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,010 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,011 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,011 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,014 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,014 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,015 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,015 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,015 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,015 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,015 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,015 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,015 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,015 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,015 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:52,015 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:52,015 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,015 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. started... 2023-08-06 09:54:52,016 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88b081660e8b4c6ad85dd3e66e468f89 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-06 09:54:52,017 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,017 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,017 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,026 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/3f72b8156de6413c9a00221c76f2766f 2023-08-06 09:54:52,038 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/6371d74289a84f30a44633e7a6de17af 2023-08-06 09:54:52,042 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/3f72b8156de6413c9a00221c76f2766f as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/3f72b8156de6413c9a00221c76f2766f 2023-08-06 09:54:52,046 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/3f72b8156de6413c9a00221c76f2766f, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:54:52,047 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/6371d74289a84f30a44633e7a6de17af as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/6371d74289a84f30a44633e7a6de17af 2023-08-06 09:54:52,051 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/6371d74289a84f30a44633e7a6de17af, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:54:52,051 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 88b081660e8b4c6ad85dd3e66e468f89 in 35ms, sequenceid=6, compaction requested=false 2023-08-06 09:54:52,052 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-08-06 09:54:52,052 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88b081660e8b4c6ad85dd3e66e468f89: 2023-08-06 09:54:52,052 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,052 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:52,052 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:52,052 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,052 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,052 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,054 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,054 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,055 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,055 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-06 09:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c95fd4c[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:52,057 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-06 09:54:52,057 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,058 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,058 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,058 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,058 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,058 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,058 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,058 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,058 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:52,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,058 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,058 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,058 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,058 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:52,058 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,059 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:52,059 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,059 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,059 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,061 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,061 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,061 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,061 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,061 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,061 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,061 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:54:52,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-06 09:54:52,064 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-06 09:54:52,064 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:52,064 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:52,064 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-06 09:54:52,064 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:52,066 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,066 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,066 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,066 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,066 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,066 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,067 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,067 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,067 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,067 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,069 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,069 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,070 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,072 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,072 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,072 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,072 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-06 09:54:52,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:52,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,072 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,072 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,072 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:52,072 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:52,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,073 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,173 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:52,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:52,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:52,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-06 09:54:52,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-06 09:54:52,187 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-06 09:54:52,188 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,188 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-06 09:54:52,188 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-06 09:54:52,188 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,188 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,189 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,189 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,189 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,189 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,190 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,190 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:52,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,190 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,190 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,190 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,190 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,191 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,191 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,191 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,191 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,191 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,191 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,191 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,191 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,191 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,191 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,191 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,191 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,191 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,191 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,191 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,191 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,191 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,191 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:52,192 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,192 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,192 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. started... 2023-08-06 09:54:52,192 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:52,192 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,192 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88b081660e8b4c6ad85dd3e66e468f89 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:54:52,192 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,192 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,193 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,193 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,193 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,197 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,197 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,197 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,202 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/52f70545db28424fa5ff1c94a7172027 2023-08-06 09:54:52,212 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/71c460c1443d47ad8f78a6dec6e7afb1 2023-08-06 09:54:52,217 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/52f70545db28424fa5ff1c94a7172027 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/52f70545db28424fa5ff1c94a7172027 2023-08-06 09:54:52,221 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/52f70545db28424fa5ff1c94a7172027, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:54:52,222 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/71c460c1443d47ad8f78a6dec6e7afb1 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/71c460c1443d47ad8f78a6dec6e7afb1 2023-08-06 09:54:52,226 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/71c460c1443d47ad8f78a6dec6e7afb1, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:54:52,227 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 88b081660e8b4c6ad85dd3e66e468f89 in 35ms, sequenceid=19, compaction requested=false 2023-08-06 09:54:52,227 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88b081660e8b4c6ad85dd3e66e468f89: 2023-08-06 09:54:52,227 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,227 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:52,227 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:52,227 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,229 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,230 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,230 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,230 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-06 09:54:52,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53fcc2a1[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:52,232 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-06 09:54:52,233 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,234 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,234 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,234 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,234 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,234 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,234 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:52,234 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,235 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:52,234 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,234 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,235 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,235 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,235 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:52,235 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:52,236 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,236 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,236 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,237 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,237 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,237 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,238 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,238 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,238 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,239 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:54:52,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-06 09:54:52,242 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-06 09:54:52,243 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:52,243 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:52,243 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-06 09:54:52,243 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:52,244 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,244 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,244 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,244 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,244 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,244 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,245 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,245 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,245 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,246 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,248 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,248 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,249 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,252 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,252 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,252 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,252 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,252 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,252 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,252 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,252 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,252 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,253 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,252 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-06 09:54:52,253 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:52,252 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,253 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,253 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,253 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,253 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,253 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,253 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,253 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:52,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,253 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:52,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,354 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:52,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:52,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:52,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-06 09:54:52,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-06 09:54:52,370 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-06 09:54:52,370 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,371 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-06 09:54:52,371 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-06 09:54:52,371 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,371 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,373 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,373 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,373 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,373 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,373 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,373 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:52,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,375 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,375 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,375 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,375 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,375 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,375 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,375 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,375 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,375 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,375 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,375 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,375 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,375 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,375 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,376 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,376 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,376 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:52,376 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:52,376 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,376 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. started... 2023-08-06 09:54:52,376 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88b081660e8b4c6ad85dd3e66e468f89 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:54:52,377 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,377 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,377 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,377 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,378 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,378 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,387 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/42eb49cf445f4e35979e4399d8b89f14 2023-08-06 09:54:52,401 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/8c8afcb90bc94f889d78285803687803 2023-08-06 09:54:52,406 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/42eb49cf445f4e35979e4399d8b89f14 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/42eb49cf445f4e35979e4399d8b89f14 2023-08-06 09:54:52,410 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/42eb49cf445f4e35979e4399d8b89f14, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:54:52,411 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/8c8afcb90bc94f889d78285803687803 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/8c8afcb90bc94f889d78285803687803 2023-08-06 09:54:52,414 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/8c8afcb90bc94f889d78285803687803, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:54:52,415 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 88b081660e8b4c6ad85dd3e66e468f89 in 39ms, sequenceid=32, compaction requested=true 2023-08-06 09:54:52,415 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88b081660e8b4c6ad85dd3e66e468f89: 2023-08-06 09:54:52,415 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,416 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:52,416 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:52,416 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,416 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,416 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,417 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,417 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,417 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,417 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-06 09:54:52,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5848c472[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:52,419 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-06 09:54:52,419 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,421 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,421 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,421 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,421 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,421 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,421 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:52,421 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,421 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:52,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,421 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,421 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,421 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,422 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,422 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:52,422 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:52,423 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,423 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,423 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,423 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,423 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,423 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,423 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,424 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,424 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,424 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:54:52,427 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-06 09:54:52,427 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-06 09:54:52,427 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:52,427 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:52,427 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-06 09:54:52,427 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:52,430 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,430 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,430 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,430 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,430 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,430 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,430 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,430 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,431 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,431 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,433 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,433 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,433 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,439 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,439 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,439 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,439 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-06 09:54:52,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:52,439 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,439 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,439 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,439 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:52,439 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,439 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,439 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:52,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,439 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,440 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,440 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,440 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,440 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,440 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,440 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,440 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,440 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,540 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:52,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:52,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:52,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-06 09:54:52,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-06 09:54:52,555 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-06 09:54:52,555 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,555 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-06 09:54:52,555 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-06 09:54:52,556 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,556 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,557 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,557 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,557 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,557 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,557 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,557 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:52,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,558 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,558 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,558 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,558 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,559 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,559 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,559 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,559 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,559 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,559 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,559 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,559 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,559 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,559 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,559 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,559 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,559 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,559 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,560 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,560 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,560 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:52,560 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,560 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,560 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,560 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:52,560 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,560 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,560 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. started... 2023-08-06 09:54:52,560 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,560 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,560 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88b081660e8b4c6ad85dd3e66e468f89 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:54:52,561 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,561 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,566 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,566 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,566 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,573 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/7bfede5d56aa4e9ab5793b848c7d8701 2023-08-06 09:54:52,589 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/b7b971b05c3c45459db1a10acab5d378 2023-08-06 09:54:52,594 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/7bfede5d56aa4e9ab5793b848c7d8701 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/7bfede5d56aa4e9ab5793b848c7d8701 2023-08-06 09:54:52,599 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/7bfede5d56aa4e9ab5793b848c7d8701, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:54:52,599 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/b7b971b05c3c45459db1a10acab5d378 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/b7b971b05c3c45459db1a10acab5d378 2023-08-06 09:54:52,603 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/b7b971b05c3c45459db1a10acab5d378, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:54:52,604 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 88b081660e8b4c6ad85dd3e66e468f89 in 44ms, sequenceid=45, compaction requested=true 2023-08-06 09:54:52,604 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88b081660e8b4c6ad85dd3e66e468f89: 2023-08-06 09:54:52,604 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,604 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:52,604 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:52,604 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,605 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,605 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,606 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,606 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,607 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,607 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-06 09:54:52,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58ca2c5f[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:52,609 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-06 09:54:52,609 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,610 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,610 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,610 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,611 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,611 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,611 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,611 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:52,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,611 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:52,611 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,611 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,611 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:52,611 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,611 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,611 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:52,612 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,612 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,612 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,614 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,614 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,614 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,614 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,614 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,614 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,614 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:54:52,617 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-06 09:54:52,617 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-06 09:54:52,617 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:52,617 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:52,617 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-06 09:54:52,617 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:52,619 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,619 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,619 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,619 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,619 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,619 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,619 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,619 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,620 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,620 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,622 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,622 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,622 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,625 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,625 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-06 09:54:52,625 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,626 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,625 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,626 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:52,626 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,726 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:52,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:52,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:52,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-06 09:54:52,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-06 09:54:52,740 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-06 09:54:52,740 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,740 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-06 09:54:52,740 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-06 09:54:52,741 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,741 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,742 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,742 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,742 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,742 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,743 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,743 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:52,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,743 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,743 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,743 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,744 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,744 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,744 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,744 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,744 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,744 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,744 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,744 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,744 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,744 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,744 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,745 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,745 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,745 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,745 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,745 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,745 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,745 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:52,745 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:52,745 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,745 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. started... 2023-08-06 09:54:52,745 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88b081660e8b4c6ad85dd3e66e468f89 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:54:52,746 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,747 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,747 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,747 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,748 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,748 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,754 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/39a3fbda98534337bd63b7e496e61a05 2023-08-06 09:54:52,765 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/8dda3899e4004cfa9d6af3765e07d190 2023-08-06 09:54:52,769 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/39a3fbda98534337bd63b7e496e61a05 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/39a3fbda98534337bd63b7e496e61a05 2023-08-06 09:54:52,772 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/39a3fbda98534337bd63b7e496e61a05, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:54:52,773 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/8dda3899e4004cfa9d6af3765e07d190 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/8dda3899e4004cfa9d6af3765e07d190 2023-08-06 09:54:52,777 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/8dda3899e4004cfa9d6af3765e07d190, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:54:52,777 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 88b081660e8b4c6ad85dd3e66e468f89 in 32ms, sequenceid=58, compaction requested=true 2023-08-06 09:54:52,777 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88b081660e8b4c6ad85dd3e66e468f89: 2023-08-06 09:54:52,777 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,777 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:52,778 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:52,778 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,778 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,778 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,779 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,779 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,779 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,780 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,781 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-06 09:54:52,781 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b76c597[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:52,781 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-06 09:54:52,781 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,782 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,782 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,782 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,782 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,783 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:52,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,783 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,783 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,783 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,783 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:52,783 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,783 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,783 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,783 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:52,783 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:52,793 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:54:52,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-06 09:54:52,796 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-06 09:54:52,796 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:52,796 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:52,796 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-06 09:54:52,796 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:52,800 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,800 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,800 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,800 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,801 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,800 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,801 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,801 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,801 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,802 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,803 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,804 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,804 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,806 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,806 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,806 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,807 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,806 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,806 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,807 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,807 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-06 09:54:52,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:52,807 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,807 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,807 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,807 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,807 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:52,807 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:52,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,807 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,808 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,808 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,808 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,808 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,808 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,908 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:52,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:52,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:52,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-06 09:54:52,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-06 09:54:52,921 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-06 09:54:52,921 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,921 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-06 09:54:52,921 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-06 09:54:52,922 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,922 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,923 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,923 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,923 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,923 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,923 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,923 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:52,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,924 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-06 09:54:52,924 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,924 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:52,925 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,925 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. started... 2023-08-06 09:54:52,925 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88b081660e8b4c6ad85dd3e66e468f89 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:54:52,927 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,928 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,928 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,928 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,928 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,928 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,934 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/c9873d5a1f3b4154be6dcda9fc3c1b29 2023-08-06 09:54:52,945 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/3cc712df9ed142069c09ceaa9731936f 2023-08-06 09:54:52,949 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/c9873d5a1f3b4154be6dcda9fc3c1b29 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/c9873d5a1f3b4154be6dcda9fc3c1b29 2023-08-06 09:54:52,952 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/c9873d5a1f3b4154be6dcda9fc3c1b29, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:54:52,953 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/3cc712df9ed142069c09ceaa9731936f as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/3cc712df9ed142069c09ceaa9731936f 2023-08-06 09:54:52,956 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/3cc712df9ed142069c09ceaa9731936f, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:54:52,957 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 88b081660e8b4c6ad85dd3e66e468f89 in 32ms, sequenceid=71, compaction requested=true 2023-08-06 09:54:52,957 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88b081660e8b4c6ad85dd3e66e468f89: 2023-08-06 09:54:52,957 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:52,957 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:52,957 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:52,957 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,957 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-06 09:54:52,957 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-06 09:54:52,959 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,959 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,959 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,959 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:52,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,961 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-06 09:54:52,961 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e2af592[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:52,961 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-06 09:54:52,961 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,962 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,962 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,962 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,962 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,962 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,962 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:52,962 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,962 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,962 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:52,962 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,962 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:52,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,963 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-06 09:54:52,963 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-06 09:54:52,963 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:52,965 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,965 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,965 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,965 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,965 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,965 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,965 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,966 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-06 09:54:52,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,966 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,966 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-06 09:54:52,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:54:52,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-06 09:54:52,969 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-06 09:54:52,970 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:52,970 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:52,970 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-06 09:54:52,970 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:52,971 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,971 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,971 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,971 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:52,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:52,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,971 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,971 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:52,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,972 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,972 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,973 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:52,973 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,975 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:52,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-06 09:54:52,976 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,976 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,979 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,979 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,979 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,979 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,979 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-06 09:54:52,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:52,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,979 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:52,979 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,979 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:52,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,979 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:52,979 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,979 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,980 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-06 09:54:52,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,980 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:52,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:52,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:52,979 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:52,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:52,980 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:52,980 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,980 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:52,980 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:52,980 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-06 09:54:52,980 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-06 09:54:52,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:52,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:53,080 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:53,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:53,088 INFO [Listener at localhost/36789] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:53,089 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.StoreUtils(129): cannot split hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/52f70545db28424fa5ff1c94a7172027 because midkey is the same as first or last row 2023-08-06 09:54:53,089 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.StoreUtils(129): cannot split hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/71c460c1443d47ad8f78a6dec6e7afb1 because midkey is the same as first or last row 2023-08-06 09:54:53,090 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.HRegion(2745): Flushing 88b081660e8b4c6ad85dd3e66e468f89 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-06 09:54:53,098 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/d6f5800e27344a65b6c534db8d3b0f8f 2023-08-06 09:54:53,120 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/49651c2f7945446c89afe3b9c10a0af2 2023-08-06 09:54:53,124 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf1/d6f5800e27344a65b6c534db8d3b0f8f as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/d6f5800e27344a65b6c534db8d3b0f8f 2023-08-06 09:54:53,128 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/d6f5800e27344a65b6c534db8d3b0f8f, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:54:53,129 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/.tmp/cf2/49651c2f7945446c89afe3b9c10a0af2 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/49651c2f7945446c89afe3b9c10a0af2 2023-08-06 09:54:53,133 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/49651c2f7945446c89afe3b9c10a0af2, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:54:53,134 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 88b081660e8b4c6ad85dd3e66e468f89 in 44ms, sequenceid=82, compaction requested=true 2023-08-06 09:54:53,135 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.HRegion(2446): Flush status journal for 88b081660e8b4c6ad85dd3e66e468f89: 2023-08-06 09:54:53,135 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.StoreUtils(129): cannot split hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/52f70545db28424fa5ff1c94a7172027 because midkey is the same as first or last row 2023-08-06 09:54:53,135 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39585] regionserver.StoreUtils(129): cannot split hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/71c460c1443d47ad8f78a6dec6e7afb1 because midkey is the same as first or last row 2023-08-06 09:54:53,135 DEBUG [Listener at localhost/36789] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:53,135 DEBUG [Listener at localhost/36789] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-08-06 09:54:53,136 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:54:53,137 INFO [Listener at localhost/36789] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-08-06 09:54:53,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-08-06 09:54:53,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:53,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-08-06 09:54:53,141 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315693140"}]},"ts":"1691315693140"} 2023-08-06 09:54:53,142 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-08-06 09:54:53,144 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-08-06 09:54:53,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=88b081660e8b4c6ad85dd3e66e468f89, UNASSIGN}] 2023-08-06 09:54:53,145 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=88b081660e8b4c6ad85dd3e66e468f89, UNASSIGN 2023-08-06 09:54:53,146 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=88b081660e8b4c6ad85dd3e66e468f89, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:53,146 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1691315693146"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315693146"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315693146"}]},"ts":"1691315693146"} 2023-08-06 09:54:53,147 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 88b081660e8b4c6ad85dd3e66e468f89, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:54:53,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-08-06 09:54:53,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:53,298 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 88b081660e8b4c6ad85dd3e66e468f89, disabling compactions & flushes 2023-08-06 09:54:53,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:53,298 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:53,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. after waiting 0 ms 2023-08-06 09:54:53,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:53,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-06 09:54:53,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:54:53,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89. 2023-08-06 09:54:53,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 88b081660e8b4c6ad85dd3e66e468f89: 2023-08-06 09:54:53,305 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:53,305 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=88b081660e8b4c6ad85dd3e66e468f89, regionState=CLOSED 2023-08-06 09:54:53,305 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1691315693305"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315693305"}]},"ts":"1691315693305"} 2023-08-06 09:54:53,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-08-06 09:54:53,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 88b081660e8b4c6ad85dd3e66e468f89, server=jenkins-hbase4.apache.org,39585,1691315596578 in 159 msec 2023-08-06 09:54:53,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-08-06 09:54:53,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=88b081660e8b4c6ad85dd3e66e468f89, UNASSIGN in 164 msec 2023-08-06 09:54:53,310 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315693310"}]},"ts":"1691315693310"} 2023-08-06 09:54:53,310 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-08-06 09:54:53,312 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-08-06 09:54:53,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 175 msec 2023-08-06 09:54:53,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-08-06 09:54:53,443 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-08-06 09:54:53,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-08-06 09:54:53,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:53,446 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:53,446 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:53,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-08-06 09:54:53,449 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:53,451 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/recovered.edits] 2023-08-06 09:54:53,455 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/39a3fbda98534337bd63b7e496e61a05 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/39a3fbda98534337bd63b7e496e61a05 2023-08-06 09:54:53,456 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/3f72b8156de6413c9a00221c76f2766f to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/3f72b8156de6413c9a00221c76f2766f 2023-08-06 09:54:53,457 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/42eb49cf445f4e35979e4399d8b89f14 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/42eb49cf445f4e35979e4399d8b89f14 2023-08-06 09:54:53,458 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/52f70545db28424fa5ff1c94a7172027 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/52f70545db28424fa5ff1c94a7172027 2023-08-06 09:54:53,459 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/7bfede5d56aa4e9ab5793b848c7d8701 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/7bfede5d56aa4e9ab5793b848c7d8701 2023-08-06 09:54:53,461 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/c9873d5a1f3b4154be6dcda9fc3c1b29 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/c9873d5a1f3b4154be6dcda9fc3c1b29 2023-08-06 09:54:53,462 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/d6f5800e27344a65b6c534db8d3b0f8f to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf1/d6f5800e27344a65b6c534db8d3b0f8f 2023-08-06 09:54:53,465 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/3cc712df9ed142069c09ceaa9731936f to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/3cc712df9ed142069c09ceaa9731936f 2023-08-06 09:54:53,466 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/49651c2f7945446c89afe3b9c10a0af2 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/49651c2f7945446c89afe3b9c10a0af2 2023-08-06 09:54:53,468 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/6371d74289a84f30a44633e7a6de17af to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/6371d74289a84f30a44633e7a6de17af 2023-08-06 09:54:53,469 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/71c460c1443d47ad8f78a6dec6e7afb1 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/71c460c1443d47ad8f78a6dec6e7afb1 2023-08-06 09:54:53,470 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/8c8afcb90bc94f889d78285803687803 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/8c8afcb90bc94f889d78285803687803 2023-08-06 09:54:53,471 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/8dda3899e4004cfa9d6af3765e07d190 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/8dda3899e4004cfa9d6af3765e07d190 2023-08-06 09:54:53,473 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/b7b971b05c3c45459db1a10acab5d378 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/cf2/b7b971b05c3c45459db1a10acab5d378 2023-08-06 09:54:53,475 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/recovered.edits/85.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89/recovered.edits/85.seqid 2023-08-06 09:54:53,476 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testInvalidSplitKey/88b081660e8b4c6ad85dd3e66e468f89 2023-08-06 09:54:53,476 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-08-06 09:54:53,478 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:53,479 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-08-06 09:54:53,482 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-08-06 09:54:53,483 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:53,483 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-08-06 09:54:53,483 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315693483"}]},"ts":"9223372036854775807"} 2023-08-06 09:54:53,485 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-08-06 09:54:53,485 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 88b081660e8b4c6ad85dd3e66e468f89, NAME => 'testInvalidSplitKey,,1691315691495.88b081660e8b4c6ad85dd3e66e468f89.', STARTKEY => '', ENDKEY => ''}] 2023-08-06 09:54:53,485 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-08-06 09:54:53,485 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691315693485"}]},"ts":"9223372036854775807"} 2023-08-06 09:54:53,486 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-08-06 09:54:53,487 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:53,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 44 msec 2023-08-06 09:54:53,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-08-06 09:54:53,548 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-08-06 09:54:53,564 INFO [Listener at localhost/36789] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=371 (was 349) - Thread LEAK? -, OpenFileDescriptor=747 (was 658) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=150 (was 146) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=2315 (was 2321) 2023-08-06 09:54:53,578 INFO [Listener at localhost/36789] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=371, OpenFileDescriptor=747, MaxFileDescriptor=60000, SystemLoadAverage=150, ProcessCount=171, AvailableMemoryMB=2314 2023-08-06 09:54:53,579 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:54:53,580 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-06 09:54:53,581 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-06 09:54:53,582 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:53,584 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-08-06 09:54:53,584 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-06 09:54:53,586 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:53,586 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426 empty. 2023-08-06 09:54:53,586 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:53,586 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-08-06 09:54:53,595 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-08-06 09:54:53,596 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => c85799256b90acec87cc21bc1fc71426, NAME => 'testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp 2023-08-06 09:54:53,603 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:54:53,603 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing c85799256b90acec87cc21bc1fc71426, disabling compactions & flushes 2023-08-06 09:54:53,603 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:53,603 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:53,603 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. after waiting 0 ms 2023-08-06 09:54:53,603 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:53,603 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:53,603 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for c85799256b90acec87cc21bc1fc71426: 2023-08-06 09:54:53,605 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-08-06 09:54:53,606 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315693606"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315693606"}]},"ts":"1691315693606"} 2023-08-06 09:54:53,607 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-06 09:54:53,607 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-06 09:54:53,608 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315693607"}]},"ts":"1691315693607"} 2023-08-06 09:54:53,608 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-08-06 09:54:53,612 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:54:53,613 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:54:53,613 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:54:53,613 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:54:53,613 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:54:53,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, ASSIGN}] 2023-08-06 09:54:53,614 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, ASSIGN 2023-08-06 09:54:53,615 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44015,1691315596630; forceNewPlan=false, retain=false 2023-08-06 09:54:53,765 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:54:53,766 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=c85799256b90acec87cc21bc1fc71426, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:53,766 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315693766"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315693766"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315693766"}]},"ts":"1691315693766"} 2023-08-06 09:54:53,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:54:53,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:53,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c85799256b90acec87cc21bc1fc71426, NAME => 'testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:54:53,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:54:53,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:53,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:54:53,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:53,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:53,926 INFO [StoreOpener-c85799256b90acec87cc21bc1fc71426-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:53,927 DEBUG [StoreOpener-c85799256b90acec87cc21bc1fc71426-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1 2023-08-06 09:54:53,927 DEBUG [StoreOpener-c85799256b90acec87cc21bc1fc71426-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1 2023-08-06 09:54:53,927 INFO [StoreOpener-c85799256b90acec87cc21bc1fc71426-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c85799256b90acec87cc21bc1fc71426 columnFamilyName cf1 2023-08-06 09:54:53,928 INFO [StoreOpener-c85799256b90acec87cc21bc1fc71426-1] regionserver.HStore(310): Store=c85799256b90acec87cc21bc1fc71426/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:54:53,928 INFO [StoreOpener-c85799256b90acec87cc21bc1fc71426-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:53,929 DEBUG [StoreOpener-c85799256b90acec87cc21bc1fc71426-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2 2023-08-06 09:54:53,929 DEBUG [StoreOpener-c85799256b90acec87cc21bc1fc71426-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2 2023-08-06 09:54:53,929 INFO [StoreOpener-c85799256b90acec87cc21bc1fc71426-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c85799256b90acec87cc21bc1fc71426 columnFamilyName cf2 2023-08-06 09:54:53,930 INFO [StoreOpener-c85799256b90acec87cc21bc1fc71426-1] regionserver.HStore(310): Store=c85799256b90acec87cc21bc1fc71426/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:54:53,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:53,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:53,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:54:53,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:53,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:54:53,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c85799256b90acec87cc21bc1fc71426; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63019085, jitterRate=-0.060942456126213074}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:54:53,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c85799256b90acec87cc21bc1fc71426: 2023-08-06 09:54:53,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426., pid=111, masterSystemTime=1691315693919 2023-08-06 09:54:53,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:53,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:53,937 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=c85799256b90acec87cc21bc1fc71426, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:53,937 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315693937"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315693937"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315693937"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315693937"}]},"ts":"1691315693937"} 2023-08-06 09:54:53,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-08-06 09:54:53,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 in 170 msec 2023-08-06 09:54:53,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-08-06 09:54:53,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, ASSIGN in 327 msec 2023-08-06 09:54:53,942 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-06 09:54:53,942 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315693942"}]},"ts":"1691315693942"} 2023-08-06 09:54:53,943 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-08-06 09:54:53,945 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-08-06 09:54:53,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 364 msec 2023-08-06 09:54:54,082 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:54,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-06 09:54:54,091 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,091 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,091 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-06 09:54:54,091 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:54:54,092 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,092 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,093 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,093 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,093 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,093 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,093 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,093 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,094 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,094 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,095 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,095 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,095 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,095 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. started... 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,096 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,096 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c85799256b90acec87cc21bc1fc71426 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-06 09:54:54,098 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,099 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,099 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,099 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,099 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,099 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,106 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/5344b8324cc243eb85550159d79a3038 2023-08-06 09:54:54,118 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/d90dcc8829d347a68040a783141ff712 2023-08-06 09:54:54,122 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/5344b8324cc243eb85550159d79a3038 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038 2023-08-06 09:54:54,125 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:54:54,126 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/d90dcc8829d347a68040a783141ff712 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712 2023-08-06 09:54:54,129 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:54:54,130 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c85799256b90acec87cc21bc1fc71426 in 34ms, sequenceid=6, compaction requested=false 2023-08-06 09:54:54,131 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c85799256b90acec87cc21bc1fc71426: 2023-08-06 09:54:54,131 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,133 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-06 09:54:54,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30dfcf71[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:54,135 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:54:54,135 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,136 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,136 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,136 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,136 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,136 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,136 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:54,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,136 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,136 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,136 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,136 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:54,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,136 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,139 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:54:54,142 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-06 09:54:54,142 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:54,142 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:54,142 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,142 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:54,144 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,144 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,144 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,144 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,144 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,144 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,144 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,144 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,145 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,145 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,147 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,147 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,147 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,149 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,149 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,149 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,149 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,149 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,150 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:54,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,150 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:54,151 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:54,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,251 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:54,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:54,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:54,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-06 09:54:54,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-06 09:54:54,264 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,264 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,264 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-06 09:54:54,264 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:54:54,264 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,264 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,266 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,266 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,266 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,266 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,266 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,266 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:54,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,267 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,267 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,268 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,268 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,268 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,268 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,268 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:54,268 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:54,268 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. started... 2023-08-06 09:54:54,269 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c85799256b90acec87cc21bc1fc71426 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:54:54,275 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,276 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,276 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,276 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,277 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,277 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,277 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/4fb53f94305e40de92663c985feacf58 2023-08-06 09:54:54,288 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a 2023-08-06 09:54:54,292 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/4fb53f94305e40de92663c985feacf58 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58 2023-08-06 09:54:54,296 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:54:54,296 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a 2023-08-06 09:54:54,300 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:54:54,301 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c85799256b90acec87cc21bc1fc71426 in 33ms, sequenceid=19, compaction requested=false 2023-08-06 09:54:54,301 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c85799256b90acec87cc21bc1fc71426: 2023-08-06 09:54:54,301 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:54,301 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:54,302 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:54,302 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,302 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,302 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,305 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,305 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,305 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,305 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-06 09:54:54,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@397ea5e1[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:54,307 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:54:54,307 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,308 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,308 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,308 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,308 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,308 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,308 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:54,309 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:54,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:54,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,309 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:54,310 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,310 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,310 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,312 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,312 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,312 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,312 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,313 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,313 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,313 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:54:54,315 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:54:54,315 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-06 09:54:54,315 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:54,315 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:54,315 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,315 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:54,317 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,317 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,317 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,317 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,317 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,317 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,317 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,317 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,318 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,318 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,346 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,347 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,347 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,349 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,349 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,349 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,350 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,349 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,349 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,350 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,350 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:54,350 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,350 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,350 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,350 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,350 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,350 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,350 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,350 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:54,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,350 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,451 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:54,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:54,463 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:54,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-06 09:54:54,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-06 09:54:54,465 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,465 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,465 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-06 09:54:54,465 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:54:54,465 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,465 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,467 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,467 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,467 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,467 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,468 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,468 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,469 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,468 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,469 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:54,469 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:54,469 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,469 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. started... 2023-08-06 09:54:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,470 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c85799256b90acec87cc21bc1fc71426 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:54:54,471 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,471 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,471 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,472 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,472 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,472 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,479 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/30572bfbfae0413597964d010fb35977 2023-08-06 09:54:54,489 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/390b285543ac47c3bf8d113840ccdcdd 2023-08-06 09:54:54,494 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/30572bfbfae0413597964d010fb35977 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977 2023-08-06 09:54:54,498 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:54:54,499 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/390b285543ac47c3bf8d113840ccdcdd as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd 2023-08-06 09:54:54,503 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:54:54,503 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c85799256b90acec87cc21bc1fc71426 in 33ms, sequenceid=32, compaction requested=true 2023-08-06 09:54:54,503 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c85799256b90acec87cc21bc1fc71426: 2023-08-06 09:54:54,504 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:54,504 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:54,504 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:54,504 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,504 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,504 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,505 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,505 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,506 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,506 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-06 09:54:54,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14099524[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:54,508 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:54:54,508 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,510 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,510 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,510 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,510 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,510 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,510 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,510 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:54,510 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:54,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,511 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,511 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,511 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,511 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,511 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:54,511 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:54,512 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,512 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,512 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,512 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,512 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,513 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,513 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,513 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,513 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,513 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:54:54,516 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-06 09:54:54,516 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:54,516 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:54,516 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,516 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:54,519 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,519 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,519 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,519 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,519 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,519 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,519 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,520 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,520 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,521 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,522 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,523 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,523 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,528 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,528 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,528 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,528 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,528 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,528 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,528 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,528 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,528 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,528 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:54,528 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,528 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,529 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,529 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,529 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,529 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:54,529 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,529 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,529 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:54,530 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,530 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,630 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:54,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:54,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:54,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-06 09:54:54,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-06 09:54:54,643 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,643 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,643 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-06 09:54:54,643 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:54:54,644 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,644 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,645 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,645 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,645 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,645 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,645 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,645 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:54,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,647 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,647 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,647 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,647 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,647 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,647 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,647 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,647 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,648 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,649 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,649 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:54,649 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:54,649 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:54,650 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. started... 2023-08-06 09:54:54,650 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c85799256b90acec87cc21bc1fc71426 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:54:54,651 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,652 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,652 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,658 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/e29c921c0782495b9ef68f3d2da7b018 2023-08-06 09:54:54,669 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/e09c827755c843fca31f8ef8c8143f74 2023-08-06 09:54:54,673 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/e29c921c0782495b9ef68f3d2da7b018 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018 2023-08-06 09:54:54,676 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:54:54,677 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/e09c827755c843fca31f8ef8c8143f74 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74 2023-08-06 09:54:54,680 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:54:54,681 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c85799256b90acec87cc21bc1fc71426 in 31ms, sequenceid=45, compaction requested=true 2023-08-06 09:54:54,681 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c85799256b90acec87cc21bc1fc71426: 2023-08-06 09:54:54,681 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:54,681 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:54,681 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:54,681 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,681 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,681 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,682 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,682 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,683 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,683 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-06 09:54:54,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6df66452[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:54,684 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:54:54,684 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,685 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,685 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,685 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,686 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,686 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,686 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:54,686 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,686 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,686 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,686 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,686 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:54,686 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:54,686 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,686 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:54,687 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,687 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,687 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,689 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,689 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,689 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,689 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,689 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,689 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,689 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:54:54,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:54:54,692 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-06 09:54:54,692 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:54,692 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:54,692 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,692 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:54,693 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,693 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,693 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,694 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,693 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,694 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,694 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,694 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,694 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,695 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,696 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,697 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,697 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,699 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,699 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,699 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,699 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,699 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,699 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,699 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,699 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:54,699 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,699 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,700 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,700 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,700 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,700 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,700 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,700 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,700 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,700 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,700 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:54,700 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:54,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,800 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:54,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:54,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:54,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-06 09:54:54,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-06 09:54:54,814 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,814 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,814 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-06 09:54:54,814 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:54:54,815 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,815 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,816 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,816 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,816 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,816 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,816 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,816 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:54,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,817 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,817 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,817 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,817 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,817 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,817 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,817 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:54,818 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:54,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,818 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. started... 2023-08-06 09:54:54,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,818 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c85799256b90acec87cc21bc1fc71426 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:54:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,820 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,821 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,821 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,821 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,822 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,822 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,829 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/36e0e437064f4ebd9e2348d1cd027825 2023-08-06 09:54:54,840 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/63038de97938447e946ff676a3ee608c 2023-08-06 09:54:54,844 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/36e0e437064f4ebd9e2348d1cd027825 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825 2023-08-06 09:54:54,848 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:54:54,849 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/63038de97938447e946ff676a3ee608c as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c 2023-08-06 09:54:54,854 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:54:54,855 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c85799256b90acec87cc21bc1fc71426 in 37ms, sequenceid=58, compaction requested=true 2023-08-06 09:54:54,855 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c85799256b90acec87cc21bc1fc71426: 2023-08-06 09:54:54,855 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:54,855 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:54,856 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:54,856 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,856 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,856 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,860 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,860 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,860 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,860 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-06 09:54:54,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@231c9ada[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:54,862 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:54:54,862 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,863 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,863 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,863 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,863 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,864 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,864 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:54,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,863 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,864 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:54,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,864 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,864 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,864 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:54,864 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:54,864 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:54,864 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:54,865 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,865 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,865 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,866 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,866 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:54,866 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,866 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,866 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,866 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,866 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:54,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:54:54,869 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:54:54,869 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-06 09:54:54,869 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:54,869 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:54,869 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,869 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:54,871 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,871 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,871 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,871 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,871 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,871 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:54,871 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,872 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,872 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,873 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,874 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:54,875 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:54,875 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,877 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,877 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,877 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:54,877 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,877 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,877 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,877 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,877 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:54,877 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:54,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,878 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:54,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,878 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:54,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,877 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,878 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,878 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,878 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,878 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,878 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:54,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:54,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:54,878 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:54,878 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,878 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,978 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:54,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:54,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:54:54,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-06 09:54:54,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-06 09:54:54,991 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:54,991 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,992 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-06 09:54:54,992 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-06 09:54:54,992 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,992 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,993 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,993 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,993 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,993 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:54,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:54,994 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,994 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:54:54,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,995 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:54:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:54:54,995 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:54,995 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. started... 2023-08-06 09:54:54,996 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c85799256b90acec87cc21bc1fc71426 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:54:54,997 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,997 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,997 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,997 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:54,997 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:54,997 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:55,004 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/8c6575a2ff2b4928a6a29ca4496f309a 2023-08-06 09:54:55,015 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd 2023-08-06 09:54:55,019 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/8c6575a2ff2b4928a6a29ca4496f309a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a 2023-08-06 09:54:55,023 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:54:55,024 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd 2023-08-06 09:54:55,027 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:54:55,028 INFO [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c85799256b90acec87cc21bc1fc71426 in 32ms, sequenceid=71, compaction requested=true 2023-08-06 09:54:55,029 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c85799256b90acec87cc21bc1fc71426: 2023-08-06 09:54:55,029 DEBUG [rs(jenkins-hbase4.apache.org,44015,1691315596630)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:55,029 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:54:55,029 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:54:55,029 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,029 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-06 09:54:55,029 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-06 09:54:55,031 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,031 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:55,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:55,031 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,032 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:54:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:55,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:55,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:55,033 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-06 09:54:55,033 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f0172f5[Count = 0] remaining members to acquire global barrier 2023-08-06 09:54:55,033 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-06 09:54:55,033 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,034 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,034 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,035 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,035 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:55,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,035 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:55,035 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:55,035 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,035 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:55,035 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:54:55,035 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-06 09:54:55,035 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-06 09:54:55,035 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:54:55,035 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:54:55,035 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:54:55,036 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,036 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:55,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:55,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:55,037 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:55,037 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:55,037 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:55,037 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:55,037 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:55,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:55,037 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-06 09:54:55,037 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:55,037 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-06 09:54:55,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:55,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:55,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:55,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:55,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:55,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:55,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:55,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:55,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,44015,1691315596630': 2023-08-06 09:54:55,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-06 09:54:55,040 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-06 09:54:55,040 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:54:55,040 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:54:55,040 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-06 09:54:55,040 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:54:55,042 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,042 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,042 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:55,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,042 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:54:55,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:54:55,042 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,043 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:55,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:55,043 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:55,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:55,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:54:55,043 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:55,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:55,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:55,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:55,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:55,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:55,044 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:55,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:54:55,044 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:55,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:55,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:55,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:54:55,046 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:55,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-06 09:54:55,046 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:55,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:55,046 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:55,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:55,051 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:55,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:55,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-06 09:54:55,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:54:55,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:55,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:55,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:54:55,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:54:55,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:55,051 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:55,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:55,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:54:55,052 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:55,052 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:54:55,052 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:54:55,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:54:55,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:54:55,052 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:54:55,052 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,052 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-06 09:54:55,152 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:54:55,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:54:55,411 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-08-06 09:54:55,412 DEBUG [Listener at localhost/36789] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:55,413 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,414 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,414 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,414 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,415 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:54:55,415 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:54:55,415 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:55,663 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,663 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:54:55,663 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:54:55,663 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:54:55,664 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-08-06 09:54:55,664 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:54:55,664 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:54:55,664 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:54:55,664 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:54:55,665 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:54:55,666 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-08-06 09:54:55,666 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:54:55,666 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:54:55,667 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:54:55,667 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:54:55,667 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:55,667 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:54:55,667 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:55,667 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:55,667 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:55,668 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:54:55,668 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:55,668 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:55,668 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:55,668 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:54:55,668 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:54:55,669 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:55,669 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:55,669 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:55,669 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:54:55,669 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:55,669 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:55,669 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:55,669 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:54:55,670 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:55,670 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:55,670 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:55,670 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:55,670 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:55,671 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:55,671 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:55,671 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:55,671 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:55,671 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:55,671 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:55,671 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,671 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:54:55,672 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,672 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,672 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,672 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-06 09:54:55,672 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:54:55,673 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:54:55,673 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315725673 2023-08-06 09:54:55,675 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,675 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:54:55,677 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:54:55,677 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=c85799256b90acec87cc21bc1fc71426, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:54:55,677 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:54:55,677 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315755677 2023-08-06 09:54:55,677 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315815677 2023-08-06 09:54:55,678 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:54:55,678 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:54:55,680 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:54:55,680 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,680 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:54:55,680 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:55,928 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,928 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:54:55,928 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:54:55,928 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:54:55,928 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:54:55,928 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:54:55,928 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:54:55,928 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:54:55,928 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:54:55,929 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:54:55,931 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-06 09:54:55,931 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:54:55,932 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:54:55,932 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:54:55,932 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:54:55,932 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:55,933 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:54:55,933 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:55,933 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:55,933 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:55,934 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:54:55,934 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:55,934 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:55,934 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:55,935 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:54:55,935 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:54:55,935 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:55,935 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:55,935 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:55,935 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:54:55,935 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:55,936 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:55,936 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:55,936 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:54:55,936 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:55,936 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:55,936 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:55,936 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:55,937 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:55,937 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:55,937 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:55,937 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:55,937 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:55,937 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:55,937 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:55,937 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,938 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:54:55,938 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,938 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,938 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,938 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-06 09:54:55,938 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:54:55,939 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:54:55,939 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315725939 2023-08-06 09:54:55,941 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:54:55,941 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,943 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:54:55,943 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=c85799256b90acec87cc21bc1fc71426, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:54:55,943 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:54:55,943 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315755943 2023-08-06 09:54:55,943 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315815943 2023-08-06 09:54:55,943 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:54:55,943 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:54:55,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN}] 2023-08-06 09:54:55,945 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:54:55,945 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:55,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:54:55,945 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:56,193 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,194 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:54:56,194 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:54:56,194 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:54:56,194 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:54:56,194 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:54:56,194 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:54:56,194 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:54:56,194 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:54:56,195 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:54:56,196 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-06 09:54:56,196 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:54:56,197 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:54:56,197 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:54:56,197 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:54:56,197 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:56,197 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:54:56,198 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:56,198 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:56,198 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:56,198 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:54:56,198 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:56,198 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:56,198 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:56,199 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:54:56,199 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:54:56,199 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:56,199 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:56,199 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:56,199 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:54:56,199 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:56,200 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:56,200 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:56,200 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:54:56,200 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:56,200 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:56,200 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,200 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,201 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:56,201 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,201 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,201 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:56,201 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:56,201 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:56,202 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:56,202 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,202 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:54:56,202 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,202 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,202 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,202 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-06 09:54:56,203 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:54:56,203 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:54:56,204 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315726204 2023-08-06 09:54:56,205 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:54:56,205 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,207 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:54:56,208 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=c85799256b90acec87cc21bc1fc71426, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:54:56,208 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:54:56,208 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315756208 2023-08-06 09:54:56,208 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315816208 2023-08-06 09:54:56,208 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:54:56,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN}] 2023-08-06 09:54:56,208 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:54:56,210 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN 2023-08-06 09:54:56,211 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=c85799256b90acec87cc21bc1fc71426, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:56,211 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315696211"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315696211"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315696211"}]},"ts":"1691315696211"} 2023-08-06 09:54:56,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:54:56,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c85799256b90acec87cc21bc1fc71426, disabling compactions & flushes 2023-08-06 09:54:56,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:56,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:56,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. after waiting 0 ms 2023-08-06 09:54:56,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:56,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing c85799256b90acec87cc21bc1fc71426 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-06 09:54:56,377 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/3bf56cad6ac84701a68ab93255e186d0 2023-08-06 09:54:56,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac 2023-08-06 09:54:56,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf1/3bf56cad6ac84701a68ab93255e186d0 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0 2023-08-06 09:54:56,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:54:56,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/.tmp/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac 2023-08-06 09:54:56,399 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:54:56,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c85799256b90acec87cc21bc1fc71426 in 36ms, sequenceid=82, compaction requested=true 2023-08-06 09:54:56,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-06 09:54:56,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:54:56,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. 2023-08-06 09:54:56,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c85799256b90acec87cc21bc1fc71426: 2023-08-06 09:54:56,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,408 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=c85799256b90acec87cc21bc1fc71426, regionState=CLOSED 2023-08-06 09:54:56,408 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315696408"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315696408"}]},"ts":"1691315696408"} 2023-08-06 09:54:56,410 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-08-06 09:54:56,410 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 in 197 msec 2023-08-06 09:54:56,411 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-08-06 09:54:56,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN in 202 msec 2023-08-06 09:54:56,413 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:54:56,413 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:54:56,413 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:56,458 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,458 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:54:56,458 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:54:56,458 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:54:56,459 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:54:56,459 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:54:56,459 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:54:56,459 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:54:56,459 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:54:56,459 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:54:56,461 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-06 09:54:56,461 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:54:56,461 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:54:56,461 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:54:56,461 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:54:56,461 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:54:56,461 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:54:56,462 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:56,462 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:54:56,462 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:56,462 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:56,462 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:56,463 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:54:56,463 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:56,463 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:56,463 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:56,464 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:54:56,464 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:54:56,464 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:56,464 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:56,464 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:56,465 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:54:56,465 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:56,465 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:56,465 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:56,466 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:54:56,466 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:56,466 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:56,466 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,467 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,467 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:56,467 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,467 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,467 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:56,467 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:56,468 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:56,468 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:56,468 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,468 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN 2023-08-06 09:54:56,468 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:56,468 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:54:56,468 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,468 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,468 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,468 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:56,469 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:56,469 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-08-06 09:54:56,469 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:54:56,470 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:54:56,470 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315726470 2023-08-06 09:54:56,471 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:54:56,472 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,474 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:54:56,474 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=c85799256b90acec87cc21bc1fc71426, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:54:56,474 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:54:56,474 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315756474 2023-08-06 09:54:56,474 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315816474 2023-08-06 09:54:56,474 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:54:56,477 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:54:56,485 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=c85799256b90acec87cc21bc1fc71426, threads=14 2023-08-06 09:54:56,486 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,486 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,486 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,486 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,486 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,486 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,487 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,487 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,487 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,487 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,487 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,487 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,487 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,488 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,678 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,685 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,711 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,718 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,723 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,723 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,735 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,736 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,747 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,747 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,749 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,755 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,757 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,758 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:56,758 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region c85799256b90acec87cc21bc1fc71426 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-06 09:54:56,761 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-06 09:54:56,800 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:54:56,800 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:54:56,800 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:56,975 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,975 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:54:56,975 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:54:56,975 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:54:56,975 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:54:56,975 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:54:56,975 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:54:56,975 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:54:56,975 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:54:56,976 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:54:56,977 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-06 09:54:56,977 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:54:56,978 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:54:56,978 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:54:56,978 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:54:56,978 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:54:56,978 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:54:56,978 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:56,979 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:54:56,979 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:56,979 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:56,979 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:56,979 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:54:56,979 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:56,979 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:56,979 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:56,980 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:54:56,980 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:54:56,980 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:56,980 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:56,980 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:56,980 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:54:56,981 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:56,981 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:56,981 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:56,981 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:54:56,981 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:56,982 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:56,982 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,982 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,982 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:56,983 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,983 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:56,983 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:56,983 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:56,983 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:56,984 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:56,984 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,984 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN 2023-08-06 09:54:56,984 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:56,984 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:54:56,985 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,985 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,985 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,985 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:56,985 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:56,985 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-06 09:54:56,986 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:54:56,986 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:54:56,987 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315726987 2023-08-06 09:54:56,989 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:56,989 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:54:56,991 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:54:56,991 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=c85799256b90acec87cc21bc1fc71426, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:54:56,991 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:54:56,992 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315756992 2023-08-06 09:54:56,992 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315816992 2023-08-06 09:54:56,992 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:54:57,001 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=c85799256b90acec87cc21bc1fc71426, threads=14 2023-08-06 09:54:57,001 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,001 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,001 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,002 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,002 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,002 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,006 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,007 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,007 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,007 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,007 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,007 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,008 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,008 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,180 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,180 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,182 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,187 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,192 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,208 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,208 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,210 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,214 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,214 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,215 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58 for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,223 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,225 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,225 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd for region: c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:57,226 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region c85799256b90acec87cc21bc1fc71426 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-06 09:54:57,260 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:54:57,265 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:54:57,267 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:54:57,267 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:54:57,267 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:54:57,267 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:57,492 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,492 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:54:57,492 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:54:57,492 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:54:57,492 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:54:57,492 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:54:57,493 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:54:57,493 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:54:57,493 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:54:57,493 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:54:57,494 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-06 09:54:57,495 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:54:57,495 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:54:57,495 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:54:57,495 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:54:57,495 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:54:57,495 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:54:57,495 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:57,496 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:54:57,496 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:57,496 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:57,496 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:57,496 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:54:57,496 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:57,496 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:57,496 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:57,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:54:57,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:54:57,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:57,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:57,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:57,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:54:57,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:57,497 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:57,498 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:57,498 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:54:57,498 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:57,498 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:57,498 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:57,498 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:57,498 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:57,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:57,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:57,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:57,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:57,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:57,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:57,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,499 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN 2023-08-06 09:54:57,500 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:57,500 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:54:57,500 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,500 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,500 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,500 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:57,500 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:57,500 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-06 09:54:57,501 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:54:57,501 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:54:57,501 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315727501 2023-08-06 09:54:57,503 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,503 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:54:57,505 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:54:57,506 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=c85799256b90acec87cc21bc1fc71426, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:54:57,506 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:54:57,506 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315757506 2023-08-06 09:54:57,506 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315817506 2023-08-06 09:54:57,506 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:54:57,508 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:54:57,510 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:54:57,510 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:54:57,510 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:57,756 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,756 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:54:57,756 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:54:57,756 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:54:57,756 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:54:57,757 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:54:57,757 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:54:57,757 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:54:57,757 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:54:57,757 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:54:57,758 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-06 09:54:57,758 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:54:57,758 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:54:57,759 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:54:57,759 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:54:57,759 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:54:57,759 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:54:57,759 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:57,760 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:54:57,760 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:57,760 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:57,760 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:57,760 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:54:57,760 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:57,760 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:57,760 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:57,761 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:54:57,761 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:54:57,761 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:57,761 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:57,761 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:57,761 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:54:57,761 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:57,761 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:57,761 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:57,762 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:54:57,762 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:57,762 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:57,762 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:57,762 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:57,762 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:57,762 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:57,762 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:57,763 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:57,763 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:57,763 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:57,763 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:57,763 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,763 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN 2023-08-06 09:54:57,763 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:57,764 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:54:57,764 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,764 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,764 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,764 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:57,764 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:57,764 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-06 09:54:57,765 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:54:57,765 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:54:57,765 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315727765 2023-08-06 09:54:57,767 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:54:57,767 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,769 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:54:57,769 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=c85799256b90acec87cc21bc1fc71426, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:54:57,769 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:54:57,769 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315757769 2023-08-06 09:54:57,769 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315817769 2023-08-06 09:54:57,770 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-06 09:54:57,770 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:54:57,772 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315697771"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1691315697771"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1691315697771"}]},"ts":"1691315697771"} 2023-08-06 09:54:57,772 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315697771"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315697771"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315697771"}]},"ts":"1691315697771"} 2023-08-06 09:54:57,772 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315697771"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315697771"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315697771"}]},"ts":"1691315697771"} 2023-08-06 09:54:57,773 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:54:57,773 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:57,773 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:54:57,773 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:58,020 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,020 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:54:58,020 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:54:58,020 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:54:58,020 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:54:58,020 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:54:58,020 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:54:58,020 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:54:58,021 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:54:58,021 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:54:58,022 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-08-06 09:54:58,022 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:54:58,023 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:54:58,023 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:54:58,023 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:54:58,024 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:54:58,024 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:54:58,024 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:58,024 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:54:58,024 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:58,025 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:58,025 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:58,025 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:54:58,025 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:58,025 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:58,026 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:58,026 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:54:58,026 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:54:58,027 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:58,027 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:58,027 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:58,027 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:54:58,027 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:58,027 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:58,027 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:58,027 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:54:58,028 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:58,028 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:58,028 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,028 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,028 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:58,029 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,029 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,029 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:58,029 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:58,029 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:58,029 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:58,029 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,029 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN 2023-08-06 09:54:58,030 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:58,030 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:54:58,030 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,030 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,030 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,030 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:58,030 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:58,030 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-06 09:54:58,031 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:54:58,031 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:54:58,031 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315728031 2023-08-06 09:54:58,033 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,033 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:54:58,035 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:54:58,035 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=c85799256b90acec87cc21bc1fc71426, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:54:58,035 INFO [Listener at localhost/36789] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,44015,1691315596630, table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:58,035 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=9ea200f882928210ab7d483fbadfc340, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-06 09:54:58,036 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-06 09:54:58,036 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:54:58,036 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315758036 2023-08-06 09:54:58,036 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315818036 2023-08-06 09:54:58,036 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:54:58,036 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1691315698036"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1691315698036"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1691315698036"}]},"ts":"1691315698036"} 2023-08-06 09:54:58,036 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315698036"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315698036"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315698036"}]},"ts":"1691315698036"} 2023-08-06 09:54:58,037 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315698036"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315698036"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315698036"}]},"ts":"1691315698036"} 2023-08-06 09:54:58,038 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:54:58,040 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:54:58,040 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:54:58,040 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:58,286 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,286 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:54:58,286 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:54:58,286 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:54:58,287 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:54:58,287 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:54:58,287 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:54:58,287 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:54:58,287 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:54:58,287 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:54:58,289 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-08-06 09:54:58,289 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:54:58,289 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:54:58,289 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:54:58,290 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:54:58,290 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:54:58,290 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:54:58,290 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:58,291 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:54:58,291 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:58,291 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:58,291 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:58,291 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:54:58,291 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:58,291 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:58,292 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:58,292 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:54:58,292 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:54:58,293 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:58,293 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:58,293 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:58,293 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:54:58,293 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:58,293 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:58,294 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:58,294 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:54:58,294 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:58,294 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:58,295 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,295 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,295 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:58,295 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,295 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,296 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:58,296 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:58,296 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:58,296 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:58,297 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,297 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN 2023-08-06 09:54:58,297 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:58,297 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:54:58,297 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,297 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,297 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,298 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:58,298 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:58,298 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-08-06 09:54:58,299 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:54:58,299 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:54:58,300 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315728300 2023-08-06 09:54:58,302 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:54:58,302 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,304 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:54:58,304 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=c85799256b90acec87cc21bc1fc71426, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:54:58,304 INFO [Listener at localhost/36789] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,44015,1691315596630, table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:58,304 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=9ea200f882928210ab7d483fbadfc340, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-06 09:54:58,304 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-06 09:54:58,304 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:54:58,305 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315758305 2023-08-06 09:54:58,305 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315818305 2023-08-06 09:54:58,305 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:54:58,305 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:54:58,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9ea200f882928210ab7d483fbadfc340, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, ASSIGN}] 2023-08-06 09:54:58,307 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:54:58,307 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:54:58,307 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:58,555 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,555 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:54:58,555 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:54:58,555 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:54:58,555 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:54:58,555 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:54:58,555 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:54:58,555 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:54:58,556 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:54:58,556 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:54:58,557 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-08-06 09:54:58,557 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:54:58,558 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:54:58,558 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:54:58,558 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:54:58,558 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:54:58,559 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:54:58,559 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:58,559 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:54:58,559 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:58,559 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:58,559 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:58,559 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:54:58,559 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:58,559 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:58,560 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:58,560 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:54:58,560 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:54:58,560 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:58,560 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:58,560 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:58,560 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:54:58,560 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:58,561 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:58,561 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:58,561 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:54:58,561 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:58,561 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:58,561 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,561 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,561 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:58,562 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,562 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:58,562 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:58,562 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:58,562 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:58,562 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:58,562 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,562 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN 2023-08-06 09:54:58,562 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:58,563 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:54:58,563 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,563 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,563 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,563 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:58,563 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:58,563 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-06 09:54:58,564 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:54:58,564 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:54:58,564 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315728564 2023-08-06 09:54:58,565 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:54:58,566 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,568 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:54:58,568 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=c85799256b90acec87cc21bc1fc71426, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:54:58,569 INFO [Listener at localhost/36789] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,44015,1691315596630, table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:58,569 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=9ea200f882928210ab7d483fbadfc340, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-06 09:54:58,569 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-06 09:54:58,569 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:54:58,569 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315758569 2023-08-06 09:54:58,569 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315818569 2023-08-06 09:54:58,569 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-06 09:54:58,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9ea200f882928210ab7d483fbadfc340, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, ASSIGN}] 2023-08-06 09:54:58,570 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:54:58,572 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, ASSIGN 2023-08-06 09:54:58,572 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,44015,1691315596630; forceNewPlan=false, retain=false 2023-08-06 09:54:58,573 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9ea200f882928210ab7d483fbadfc340, ASSIGN 2023-08-06 09:54:58,573 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9ea200f882928210ab7d483fbadfc340, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,44015,1691315596630; forceNewPlan=false, retain=false 2023-08-06 09:54:58,722 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-06 09:54:58,724 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:58,724 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315698724"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315698724"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315698724"}]},"ts":"1691315698724"} 2023-08-06 09:54:58,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure 0cd9b1c17cb3ffec98c8d2ae60ed4eb8, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:54:58,728 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=9ea200f882928210ab7d483fbadfc340, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:58,728 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315698728"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315698728"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315698728"}]},"ts":"1691315698728"} 2023-08-06 09:54:58,729 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure 9ea200f882928210ab7d483fbadfc340, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:54:58,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. 2023-08-06 09:54:58,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0cd9b1c17cb3ffec98c8d2ae60ed4eb8, NAME => 'testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.', STARTKEY => '41', ENDKEY => ''} 2023-08-06 09:54:58,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:54:58,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:54:58,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,881 INFO [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,882 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1 2023-08-06 09:54:58,882 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1 2023-08-06 09:54:58,882 INFO [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 columnFamilyName cf1 2023-08-06 09:54:58,894 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977-top 2023-08-06 09:54:58,897 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825-top 2023-08-06 09:54:58,901 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0-top 2023-08-06 09:54:58,904 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58-top 2023-08-06 09:54:58,907 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038-top 2023-08-06 09:54:58,909 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a-top 2023-08-06 09:54:58,912 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018-top 2023-08-06 09:54:58,913 INFO [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(310): Store=0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:54:58,913 INFO [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,913 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2 2023-08-06 09:54:58,914 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2 2023-08-06 09:54:58,914 INFO [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 columnFamilyName cf2 2023-08-06 09:54:58,925 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd-top 2023-08-06 09:54:58,928 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a-top 2023-08-06 09:54:58,931 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c-top 2023-08-06 09:54:58,934 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd-top 2023-08-06 09:54:58,938 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712-top 2023-08-06 09:54:58,941 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac-top 2023-08-06 09:54:58,945 DEBUG [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74-top 2023-08-06 09:54:58,945 INFO [StoreOpener-0cd9b1c17cb3ffec98c8d2ae60ed4eb8-1] regionserver.HStore(310): Store=0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:54:58,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:54:58,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:58,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0cd9b1c17cb3ffec98c8d2ae60ed4eb8; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66352346, jitterRate=-0.011272996664047241}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:54:58,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0cd9b1c17cb3ffec98c8d2ae60ed4eb8: 2023-08-06 09:54:58,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8., pid=117, masterSystemTime=1691315698877 2023-08-06 09:54:58,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:54:58,953 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:54:58,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-06 09:54:58,954 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:54:58,954 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. 2023-08-06 09:54:58,954 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1912): 0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2 is initiating minor compaction (all files) 2023-08-06 09:54:58,954 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. 2023-08-06 09:54:58,954 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2 in testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. 2023-08-06 09:54:58,954 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1912): 0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1 is initiating minor compaction (all files) 2023-08-06 09:54:58,954 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1 in testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. 2023-08-06 09:54:58,954 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac-top] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/.tmp, totalSize=34.9 K 2023-08-06 09:54:58,954 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0-top] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/.tmp, totalSize=34.9 K 2023-08-06 09:54:58,955 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691315694085 2023-08-06 09:54:58,955 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691315694085 2023-08-06 09:54:58,956 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691315694252 2023-08-06 09:54:58,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. 2023-08-06 09:54:58,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. 2023-08-06 09:54:58,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. 2023-08-06 09:54:58,956 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:58,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9ea200f882928210ab7d483fbadfc340, NAME => 'testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.', STARTKEY => '', ENDKEY => '41'} 2023-08-06 09:54:58,956 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315698956"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315698956"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315698956"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315698956"}]},"ts":"1691315698956"} 2023-08-06 09:54:58,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:54:58,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:58,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:54:58,957 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691315694452 2023-08-06 09:54:58,957 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691315694252 2023-08-06 09:54:58,957 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691315694631 2023-08-06 09:54:58,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:58,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:58,958 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691315694452 2023-08-06 09:54:58,959 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691315694802 2023-08-06 09:54:58,960 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691315694631 2023-08-06 09:54:58,960 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691315694980 2023-08-06 09:54:58,960 INFO [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:58,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-08-06 09:54:58,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 0cd9b1c17cb3ffec98c8d2ae60ed4eb8, server=jenkins-hbase4.apache.org,44015,1691315596630 in 233 msec 2023-08-06 09:54:58,961 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691315694802 2023-08-06 09:54:58,961 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691315694980 2023-08-06 09:54:58,961 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691315695154 2023-08-06 09:54:58,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, ASSIGN in 391 msec 2023-08-06 09:54:58,962 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691315695154 2023-08-06 09:54:58,962 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1 2023-08-06 09:54:58,962 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1 2023-08-06 09:54:58,963 INFO [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9ea200f882928210ab7d483fbadfc340 columnFamilyName cf1 2023-08-06 09:54:58,975 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] throttle.PressureAwareThroughputController(145): 0cd9b1c17cb3ffec98c8d2ae60ed4eb8#cf2#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:54:58,975 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0cd9b1c17cb3ffec98c8d2ae60ed4eb8#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:54:58,983 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977-bottom 2023-08-06 09:54:58,986 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825-bottom 2023-08-06 09:54:58,991 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0-bottom 2023-08-06 09:54:58,994 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58-bottom 2023-08-06 09:54:59,008 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038-bottom 2023-08-06 09:54:59,010 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/.tmp/cf1/4b635c31be7d4e76bbf9dbfaff771eb9 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/4b635c31be7d4e76bbf9dbfaff771eb9 2023-08-06 09:54:59,012 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a-bottom 2023-08-06 09:54:59,014 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/.tmp/cf2/6b5a4ce3a1554587aab10414f7642c2a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/6b5a4ce3a1554587aab10414f7642c2a 2023-08-06 09:54:59,016 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018-bottom 2023-08-06 09:54:59,016 INFO [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(310): Store=9ea200f882928210ab7d483fbadfc340/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:54:59,016 INFO [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:59,017 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1 of 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 into 4b635c31be7d4e76bbf9dbfaff771eb9(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:54:59,017 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0cd9b1c17cb3ffec98c8d2ae60ed4eb8: 2023-08-06 09:54:59,017 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8., storeName=0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1, priority=9, startTime=1691315698951; duration=0sec 2023-08-06 09:54:59,017 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:54:59,018 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2 2023-08-06 09:54:59,018 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2 2023-08-06 09:54:59,018 INFO [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9ea200f882928210ab7d483fbadfc340 columnFamilyName cf2 2023-08-06 09:54:59,020 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2 of 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 into 6b5a4ce3a1554587aab10414f7642c2a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:54:59,020 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0cd9b1c17cb3ffec98c8d2ae60ed4eb8: 2023-08-06 09:54:59,020 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8., storeName=0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2, priority=9, startTime=1691315698953; duration=0sec 2023-08-06 09:54:59,020 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:54:59,029 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd-bottom 2023-08-06 09:54:59,032 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a-bottom 2023-08-06 09:54:59,035 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c-bottom 2023-08-06 09:54:59,038 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd-bottom 2023-08-06 09:54:59,043 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712-bottom 2023-08-06 09:54:59,044 ERROR [BP-1515658381-172.31.14.131-1691315591457 heartbeating to localhost/127.0.0.1:43345] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1515658381-172.31.14.131-1691315591457 (Datanode Uuid d309b271-e4de-4313-87b3-7ace32969fdc) service to localhost/127.0.0.1:43345 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-06 09:54:59,046 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac-bottom 2023-08-06 09:54:59,052 DEBUG [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74-bottom 2023-08-06 09:54:59,052 INFO [StoreOpener-9ea200f882928210ab7d483fbadfc340-1] regionserver.HStore(310): Store=9ea200f882928210ab7d483fbadfc340/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:54:59,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:59,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:59,055 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:54:59,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:59,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9ea200f882928210ab7d483fbadfc340; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74638870, jitterRate=0.11220583319664001}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:54:59,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9ea200f882928210ab7d483fbadfc340: 2023-08-06 09:54:59,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340., pid=118, masterSystemTime=1691315698877 2023-08-06 09:54:59,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:54:59,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-06 09:54:59,059 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:54:59,059 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:54:59,059 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. 2023-08-06 09:54:59,059 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1912): 9ea200f882928210ab7d483fbadfc340/cf1 is initiating minor compaction (all files) 2023-08-06 09:54:59,059 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. 2023-08-06 09:54:59,059 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1912): 9ea200f882928210ab7d483fbadfc340/cf2 is initiating minor compaction (all files) 2023-08-06 09:54:59,059 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 9ea200f882928210ab7d483fbadfc340/cf1 in testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. 2023-08-06 09:54:59,059 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 9ea200f882928210ab7d483fbadfc340/cf2 in testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. 2023-08-06 09:54:59,059 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0-bottom] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/.tmp, totalSize=34.9 K 2023-08-06 09:54:59,059 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac-bottom] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/.tmp, totalSize=34.9 K 2023-08-06 09:54:59,060 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315694085 2023-08-06 09:54:59,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. 2023-08-06 09:54:59,060 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315694085 2023-08-06 09:54:59,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. 2023-08-06 09:54:59,061 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=9ea200f882928210ab7d483fbadfc340, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:59,061 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315699061"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315699061"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315699061"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315699061"}]},"ts":"1691315699061"} 2023-08-06 09:54:59,062 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315694252 2023-08-06 09:54:59,062 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315694252 2023-08-06 09:54:59,062 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315694452 2023-08-06 09:54:59,062 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315694452 2023-08-06 09:54:59,063 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315694631 2023-08-06 09:54:59,063 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315694631 2023-08-06 09:54:59,063 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315694802 2023-08-06 09:54:59,063 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting 63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315694802 2023-08-06 09:54:59,063 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315694980 2023-08-06 09:54:59,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-08-06 09:54:59,064 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315694980 2023-08-06 09:54:59,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 9ea200f882928210ab7d483fbadfc340, server=jenkins-hbase4.apache.org,44015,1691315596630 in 333 msec 2023-08-06 09:54:59,064 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] compactions.Compactor(207): Compacting 3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315695154 2023-08-06 09:54:59,064 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] compactions.Compactor(207): Compacting dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315695154 2023-08-06 09:54:59,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-08-06 09:54:59,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9ea200f882928210ab7d483fbadfc340, ASSIGN in 494 msec 2023-08-06 09:54:59,065 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-06 09:54:59,065 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:59,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:54:59,065 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-06 09:54:59,070 INFO [Listener at localhost/36789] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:59,070 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-06 09:54:59,070 INFO [Listener at localhost/36789] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:54:59,070 INFO [Listener at localhost/36789] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:54:59,070 ERROR [Listener at localhost/36789] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-06 09:54:59,070 INFO [Listener at localhost/36789] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-06 09:54:59,070 INFO [Listener at localhost/36789] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-06 09:54:59,071 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-06 09:54:59,071 INFO [Listener at localhost/36789] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-06 09:54:59,071 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-06 09:54:59,073 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-08-06 09:54:59,073 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-06 09:54:59,073 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-06 09:54:59,073 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-08-06 09:54:59,073 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-08-06 09:54:59,076 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-08-06 09:54:59,076 DEBUG [Listener at localhost/36789] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-08-06 09:54:59,077 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-06 09:54:59,077 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-06 09:54:59,077 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-06 09:54:59,077 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-06 09:54:59,078 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:59,078 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=c13a71834623cd8c0cd6adbeac46b34e, daughterA=ffe855bd6b405f1f25462543a843a917, daughterB=7352fe00040842926aae961d72c3f889 2023-08-06 09:54:59,078 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:59,078 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-06 09:54:59,078 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:59,079 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=733289ef46def4ea49c9f7f8372be558, daughterA=dd6855342e402ecbf93bd85c5d1bcd99, daughterB=01989eef194238353eaba1e7788e719d 2023-08-06 09:54:59,079 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:59,079 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-06 09:54:59,079 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] throttle.PressureAwareThroughputController(145): 9ea200f882928210ab7d483fbadfc340#cf2#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:54:59,079 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:59,079 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=daf817c4f515131ebc48eee8503ceb30, daughterA=1c3299eae6e5d57111ef71f146bf1e79, daughterB=25e7cf8cc7d7a6145f823f228184303d 2023-08-06 09:54:59,080 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=daf817c4f515131ebc48eee8503ceb30, ASSIGN 2023-08-06 09:54:59,080 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:59,080 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-06 09:54:59,080 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:59,080 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=76d3a54f2b27e12af5a7612811d713b7, daughterA=5fb8d19bc752087bfea414e375a9b7aa, daughterB=e116f9f054275a43cbb613a92a2159b1 2023-08-06 09:54:59,080 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:59,080 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-06 09:54:59,081 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:59,081 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=c7bf488ae570a703d8494b8ebc889fae, daughterA=b6ad8f165ee195bcbbe04c83a68f9d95, daughterB=c8dd2ff746c9d5cec67b26ba8e7dcef4 2023-08-06 09:54:59,081 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:59,081 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-06 09:54:59,081 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:59,082 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:59,082 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,40847,1691315596668, table=testRollbackForSplitTableRegionWithReplica, region=070e4b344b2814c926ad7094797c209d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74bda565887a7bc02c6c0743649997a4, daughterA=d98d697c48f957810db2cf2503b5f6af, daughterB=23619c3761d77deee3853cee0b079698 2023-08-06 09:54:59,082 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:59,082 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-06 09:54:59,082 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:59,083 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:59,083 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-06 09:54:59,083 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:59,083 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9ea200f882928210ab7d483fbadfc340#cf1#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:54:59,084 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:59,084 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN 2023-08-06 09:54:59,084 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:59,085 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9ea200f882928210ab7d483fbadfc340, ASSIGN 2023-08-06 09:54:59,085 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, ASSIGN 2023-08-06 09:54:59,085 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 0cd9b1c17cb3ffec98c8d2ae60ed4eb8, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:59,085 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 9ea200f882928210ab7d483fbadfc340, server=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:59,085 DEBUG [Listener at localhost/36789] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 held the lock before restarting, call acquireLock to restore it. 2023-08-06 09:54:59,085 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:59,085 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:59,085 INFO [Listener at localhost/36789] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:59,085 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:59,085 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c85799256b90acec87cc21bc1fc71426, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:59,085 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9ea200f882928210ab7d483fbadfc340, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:59,085 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:59,086 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 0cd9b1c17cb3ffec98c8d2ae60ed4eb8, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:59,086 DEBUG [Listener at localhost/36789] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 9ea200f882928210ab7d483fbadfc340, server=jenkins-hbase4.apache.org,44015,1691315596630 didn't hold the lock before restarting, skip acquiring lock. 2023-08-06 09:54:59,086 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 14 msec 2023-08-06 09:54:59,087 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-06 09:54:59,088 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,44015,1691315596630, table=hbase:meta, region=1588230740 2023-08-06 09:54:59,093 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691315729093 2023-08-06 09:54:59,094 DEBUG [Listener at localhost/36789] assignment.AssignmentManager(1517): Joining cluster... 2023-08-06 09:54:59,095 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:59,097 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=ebbcd508c2d5b7b6263f7d0763491964, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40847,1691315596668, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668, openSeqNum=2 2023-08-06 09:54:59,098 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=c85799256b90acec87cc21bc1fc71426, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=2 2023-08-06 09:54:59,098 INFO [Listener at localhost/36789] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,44015,1691315596630, table=testRecoveryAndDoubleExecution, region=c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,098 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=9ea200f882928210ab7d483fbadfc340, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=86 2023-08-06 09:54:59,098 DEBUG [Listener at localhost/36789] assignment.RegionStateStore(147): Load hbase:meta entry region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44015,1691315596630, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630, openSeqNum=86 2023-08-06 09:54:59,098 INFO [Listener at localhost/36789] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-06 09:54:59,098 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691315759098 2023-08-06 09:54:59,098 INFO [Listener at localhost/36789] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691315819098 2023-08-06 09:54:59,098 INFO [Listener at localhost/36789] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-06 09:54:59,101 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-06 09:54:59,105 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c85799256b90acec87cc21bc1fc71426, daughterA=9ea200f882928210ab7d483fbadfc340, daughterB=0cd9b1c17cb3ffec98c8d2ae60ed4eb8 in 3.6890 sec 2023-08-06 09:54:59,112 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/.tmp/cf2/4d12ec22af0f4502bcebc1b736a72973 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/4d12ec22af0f4502bcebc1b736a72973 2023-08-06 09:54:59,112 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/.tmp/cf1/8369daddd1e04d5da1a39600fbcd3a1a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/8369daddd1e04d5da1a39600fbcd3a1a 2023-08-06 09:54:59,116 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9ea200f882928210ab7d483fbadfc340/cf2 of 9ea200f882928210ab7d483fbadfc340 into 4d12ec22af0f4502bcebc1b736a72973(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:54:59,116 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9ea200f882928210ab7d483fbadfc340: 2023-08-06 09:54:59,116 INFO [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340., storeName=9ea200f882928210ab7d483fbadfc340/cf2, priority=9, startTime=1691315699057; duration=0sec 2023-08-06 09:54:59,116 DEBUG [RS:1;jenkins-hbase4:44015-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:54:59,116 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9ea200f882928210ab7d483fbadfc340/cf1 of 9ea200f882928210ab7d483fbadfc340 into 8369daddd1e04d5da1a39600fbcd3a1a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:54:59,117 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9ea200f882928210ab7d483fbadfc340: 2023-08-06 09:54:59,117 INFO [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340., storeName=9ea200f882928210ab7d483fbadfc340/cf1, priority=9, startTime=1691315699057; duration=0sec 2023-08-06 09:54:59,117 DEBUG [RS:1;jenkins-hbase4:44015-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:54:59,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44015] ipc.CallRunner(144): callId: 862 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:54744 deadline: 1691315759352, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426. is not online on jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:59,463 DEBUG [Listener at localhost/36789] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-06 09:54:59,469 INFO [Listener at localhost/36789] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-06 09:54:59,470 DEBUG [Listener at localhost/36789] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-06 09:54:59,477 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:54:59,479 INFO [Listener at localhost/36789] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-08-06 09:54:59,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-08-06 09:54:59,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:59,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-08-06 09:54:59,482 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315699482"}]},"ts":"1691315699482"} 2023-08-06 09:54:59,483 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-08-06 09:54:59,484 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-08-06 09:54:59,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9ea200f882928210ab7d483fbadfc340, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, UNASSIGN}] 2023-08-06 09:54:59,485 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, UNASSIGN 2023-08-06 09:54:59,486 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:59,486 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315699486"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315699486"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315699486"}]},"ts":"1691315699486"} 2023-08-06 09:54:59,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 0cd9b1c17cb3ffec98c8d2ae60ed4eb8, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:54:59,488 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9ea200f882928210ab7d483fbadfc340, UNASSIGN 2023-08-06 09:54:59,489 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=9ea200f882928210ab7d483fbadfc340, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:54:59,489 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315699489"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315699489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315699489"}]},"ts":"1691315699489"} 2023-08-06 09:54:59,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 9ea200f882928210ab7d483fbadfc340, server=jenkins-hbase4.apache.org,44015,1691315596630}] 2023-08-06 09:54:59,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-08-06 09:54:59,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:59,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0cd9b1c17cb3ffec98c8d2ae60ed4eb8, disabling compactions & flushes 2023-08-06 09:54:59,639 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. 2023-08-06 09:54:59,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. 2023-08-06 09:54:59,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. after waiting 0 ms 2023-08-06 09:54:59,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. 2023-08-06 09:54:59,640 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0-top] to archive 2023-08-06 09:54:59,640 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:54:59,642 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,643 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,644 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,645 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,646 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,647 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,648 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,649 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac-top] to archive 2023-08-06 09:54:59,650 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:54:59,651 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,652 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,653 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,654 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,655 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,656 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,656 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,659 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-06 09:54:59,660 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:54:59,660 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8. 2023-08-06 09:54:59,660 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0cd9b1c17cb3ffec98c8d2ae60ed4eb8: 2023-08-06 09:54:59,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:59,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:59,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9ea200f882928210ab7d483fbadfc340, disabling compactions & flushes 2023-08-06 09:54:59,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. 2023-08-06 09:54:59,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. 2023-08-06 09:54:59,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. after waiting 0 ms 2023-08-06 09:54:59,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. 2023-08-06 09:54:59,663 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, regionState=CLOSED 2023-08-06 09:54:59,663 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315699663"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315699663"}]},"ts":"1691315699663"} 2023-08-06 09:54:59,664 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0-bottom] to archive 2023-08-06 09:54:59,664 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:54:59,666 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-08-06 09:54:59,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 0cd9b1c17cb3ffec98c8d2ae60ed4eb8, server=jenkins-hbase4.apache.org,44015,1691315596630 in 177 msec 2023-08-06 09:54:59,667 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0cd9b1c17cb3ffec98c8d2ae60ed4eb8, UNASSIGN in 181 msec 2023-08-06 09:54:59,668 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,668 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,669 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,670 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,672 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,673 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac-bottom] to archive 2023-08-06 09:54:59,674 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:54:59,675 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,676 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,677 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,677 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,679 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,680 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,680 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-06 09:54:59,685 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:54:59,685 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340. 2023-08-06 09:54:59,685 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9ea200f882928210ab7d483fbadfc340: 2023-08-06 09:54:59,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:59,687 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=9ea200f882928210ab7d483fbadfc340, regionState=CLOSED 2023-08-06 09:54:59,687 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1691315699687"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315699687"}]},"ts":"1691315699687"} 2023-08-06 09:54:59,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-08-06 09:54:59,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 9ea200f882928210ab7d483fbadfc340, server=jenkins-hbase4.apache.org,44015,1691315596630 in 198 msec 2023-08-06 09:54:59,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-08-06 09:54:59,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9ea200f882928210ab7d483fbadfc340, UNASSIGN in 205 msec 2023-08-06 09:54:59,693 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315699692"}]},"ts":"1691315699692"} 2023-08-06 09:54:59,694 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-08-06 09:54:59,696 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-08-06 09:54:59,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 218 msec 2023-08-06 09:54:59,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-08-06 09:54:59,784 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-08-06 09:54:59,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-08-06 09:54:59,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:59,788 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:59,789 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:59,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-08-06 09:54:59,793 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,793 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:59,793 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:59,795 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/recovered.edits] 2023-08-06 09:54:59,796 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/recovered.edits] 2023-08-06 09:54:59,796 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/recovered.edits] 2023-08-06 09:54:59,800 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/.regioninfo to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/.regioninfo 2023-08-06 09:54:59,800 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/30572bfbfae0413597964d010fb35977 2023-08-06 09:54:59,800 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/.regioninfo to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/.regioninfo 2023-08-06 09:54:59,802 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/36e0e437064f4ebd9e2348d1cd027825 2023-08-06 09:54:59,803 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,803 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/3bf56cad6ac84701a68ab93255e186d0 2023-08-06 09:54:59,803 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/30572bfbfae0413597964d010fb35977.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,804 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/4fb53f94305e40de92663c985feacf58 2023-08-06 09:54:59,805 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,805 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/36e0e437064f4ebd9e2348d1cd027825.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,807 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/5344b8324cc243eb85550159d79a3038 2023-08-06 09:54:59,807 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,807 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/3bf56cad6ac84701a68ab93255e186d0.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,808 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/8c6575a2ff2b4928a6a29ca4496f309a 2023-08-06 09:54:59,809 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,809 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/4fb53f94305e40de92663c985feacf58.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,810 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf1/e29c921c0782495b9ef68f3d2da7b018 2023-08-06 09:54:59,810 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,810 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/5344b8324cc243eb85550159d79a3038.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,812 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,812 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/8c6575a2ff2b4928a6a29ca4496f309a.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,813 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/390b285543ac47c3bf8d113840ccdcdd 2023-08-06 09:54:59,813 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,813 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/e29c921c0782495b9ef68f3d2da7b018.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,814 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a 2023-08-06 09:54:59,816 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/63038de97938447e946ff676a3ee608c 2023-08-06 09:54:59,817 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,819 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/390b285543ac47c3bf8d113840ccdcdd.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,820 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd 2023-08-06 09:54:59,821 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,821 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/4fc00a8ab48f44fb9bdf0b6091e9c92a.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,822 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/d90dcc8829d347a68040a783141ff712 2023-08-06 09:54:59,823 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,823 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/63038de97938447e946ff676a3ee608c.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,824 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac 2023-08-06 09:54:59,825 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,826 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/c0a44d9a5b644d1c84efd6da08f1d6dd.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,826 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/cf2/e09c827755c843fca31f8ef8c8143f74 2023-08-06 09:54:59,827 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,828 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/d90dcc8829d347a68040a783141ff712.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,829 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,829 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/dedf86bc8fc24eabbeb1fdeef29c5cac.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,830 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/recovered.edits/85.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426/recovered.edits/85.seqid 2023-08-06 09:54:59,830 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,831 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/9ea200f882928210ab7d483fbadfc340/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,831 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/e09c827755c843fca31f8ef8c8143f74.c85799256b90acec87cc21bc1fc71426 2023-08-06 09:54:59,833 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/8369daddd1e04d5da1a39600fbcd3a1a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf1/8369daddd1e04d5da1a39600fbcd3a1a 2023-08-06 09:54:59,833 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/4b635c31be7d4e76bbf9dbfaff771eb9 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf1/4b635c31be7d4e76bbf9dbfaff771eb9 2023-08-06 09:54:59,835 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/4d12ec22af0f4502bcebc1b736a72973 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/cf2/4d12ec22af0f4502bcebc1b736a72973 2023-08-06 09:54:59,836 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/6b5a4ce3a1554587aab10414f7642c2a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/cf2/6b5a4ce3a1554587aab10414f7642c2a 2023-08-06 09:54:59,838 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/recovered.edits/90.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340/recovered.edits/90.seqid 2023-08-06 09:54:59,838 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/recovered.edits/90.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8/recovered.edits/90.seqid 2023-08-06 09:54:59,838 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/9ea200f882928210ab7d483fbadfc340 2023-08-06 09:54:59,839 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testRecoveryAndDoubleExecution/0cd9b1c17cb3ffec98c8d2ae60ed4eb8 2023-08-06 09:54:59,839 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-08-06 09:54:59,841 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:59,842 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-08-06 09:54:59,844 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-08-06 09:54:59,845 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:59,845 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-08-06 09:54:59,845 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315699845"}]},"ts":"9223372036854775807"} 2023-08-06 09:54:59,845 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315699845"}]},"ts":"9223372036854775807"} 2023-08-06 09:54:59,845 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315699845"}]},"ts":"9223372036854775807"} 2023-08-06 09:54:59,847 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-06 09:54:59,847 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c85799256b90acec87cc21bc1fc71426, NAME => 'testRecoveryAndDoubleExecution,,1691315693581.c85799256b90acec87cc21bc1fc71426.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 9ea200f882928210ab7d483fbadfc340, NAME => 'testRecoveryAndDoubleExecution,,1691315695412.9ea200f882928210ab7d483fbadfc340.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 0cd9b1c17cb3ffec98c8d2ae60ed4eb8, NAME => 'testRecoveryAndDoubleExecution,41,1691315695412.0cd9b1c17cb3ffec98c8d2ae60ed4eb8.', STARTKEY => '41', ENDKEY => ''}] 2023-08-06 09:54:59,847 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-08-06 09:54:59,847 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691315699847"}]},"ts":"9223372036854775807"} 2023-08-06 09:54:59,848 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-08-06 09:54:59,850 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-06 09:54:59,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 65 msec 2023-08-06 09:54:59,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-08-06 09:54:59,890 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-08-06 09:54:59,905 INFO [Listener at localhost/36789] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=378 (was 371) - Thread LEAK? -, OpenFileDescriptor=835 (was 747) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=146 (was 150), ProcessCount=171 (was 171), AvailableMemoryMB=2281 (was 2314) 2023-08-06 09:54:59,918 INFO [Listener at localhost/36789] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=378, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=146, ProcessCount=171, AvailableMemoryMB=2281 2023-08-06 09:54:59,919 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:54:59,920 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-06 09:54:59,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-06 09:54:59,922 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-06 09:54:59,924 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-08-06 09:54:59,924 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-06 09:54:59,926 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:54:59,926 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c empty. 2023-08-06 09:54:59,927 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:54:59,927 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-08-06 09:54:59,936 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-08-06 09:54:59,937 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => c0584e08f14357669d56c7fddc2d3b3c, NAME => 'testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp 2023-08-06 09:54:59,943 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:54:59,944 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing c0584e08f14357669d56c7fddc2d3b3c, disabling compactions & flushes 2023-08-06 09:54:59,944 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:54:59,944 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:54:59,944 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. after waiting 0 ms 2023-08-06 09:54:59,944 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:54:59,944 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:54:59,944 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for c0584e08f14357669d56c7fddc2d3b3c: 2023-08-06 09:54:59,945 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-08-06 09:54:59,946 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315699946"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315699946"}]},"ts":"1691315699946"} 2023-08-06 09:54:59,947 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-06 09:54:59,948 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-06 09:54:59,948 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315699948"}]},"ts":"1691315699948"} 2023-08-06 09:54:59,949 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-08-06 09:54:59,952 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:54:59,952 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:54:59,952 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:54:59,952 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:54:59,952 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:54:59,953 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c0584e08f14357669d56c7fddc2d3b3c, ASSIGN}] 2023-08-06 09:54:59,954 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c0584e08f14357669d56c7fddc2d3b3c, ASSIGN 2023-08-06 09:54:59,954 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c0584e08f14357669d56c7fddc2d3b3c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39585,1691315596578; forceNewPlan=false, retain=false 2023-08-06 09:55:00,104 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:55:00,106 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=c0584e08f14357669d56c7fddc2d3b3c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,106 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315700105"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315700105"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315700105"}]},"ts":"1691315700105"} 2023-08-06 09:55:00,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure c0584e08f14357669d56c7fddc2d3b3c, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:55:00,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:00,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c0584e08f14357669d56c7fddc2d3b3c, NAME => 'testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:55:00,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:55:00,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:00,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:55:00,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:00,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:00,263 INFO [StoreOpener-c0584e08f14357669d56c7fddc2d3b3c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:00,264 DEBUG [StoreOpener-c0584e08f14357669d56c7fddc2d3b3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1 2023-08-06 09:55:00,264 DEBUG [StoreOpener-c0584e08f14357669d56c7fddc2d3b3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1 2023-08-06 09:55:00,264 INFO [StoreOpener-c0584e08f14357669d56c7fddc2d3b3c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0584e08f14357669d56c7fddc2d3b3c columnFamilyName cf1 2023-08-06 09:55:00,265 INFO [StoreOpener-c0584e08f14357669d56c7fddc2d3b3c-1] regionserver.HStore(310): Store=c0584e08f14357669d56c7fddc2d3b3c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:00,265 INFO [StoreOpener-c0584e08f14357669d56c7fddc2d3b3c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:00,266 DEBUG [StoreOpener-c0584e08f14357669d56c7fddc2d3b3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2 2023-08-06 09:55:00,266 DEBUG [StoreOpener-c0584e08f14357669d56c7fddc2d3b3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2 2023-08-06 09:55:00,266 INFO [StoreOpener-c0584e08f14357669d56c7fddc2d3b3c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0584e08f14357669d56c7fddc2d3b3c columnFamilyName cf2 2023-08-06 09:55:00,266 INFO [StoreOpener-c0584e08f14357669d56c7fddc2d3b3c-1] regionserver.HStore(310): Store=c0584e08f14357669d56c7fddc2d3b3c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:00,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:00,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:00,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:55:00,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:00,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:55:00,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c0584e08f14357669d56c7fddc2d3b3c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60435440, jitterRate=-0.0994417667388916}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:55:00,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c0584e08f14357669d56c7fddc2d3b3c: 2023-08-06 09:55:00,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c., pid=127, masterSystemTime=1691315700258 2023-08-06 09:55:00,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:00,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:00,273 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=c0584e08f14357669d56c7fddc2d3b3c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,273 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315700273"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315700273"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315700273"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315700273"}]},"ts":"1691315700273"} 2023-08-06 09:55:00,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-08-06 09:55:00,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure c0584e08f14357669d56c7fddc2d3b3c, server=jenkins-hbase4.apache.org,39585,1691315596578 in 168 msec 2023-08-06 09:55:00,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-08-06 09:55:00,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c0584e08f14357669d56c7fddc2d3b3c, ASSIGN in 323 msec 2023-08-06 09:55:00,278 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-06 09:55:00,278 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315700278"}]},"ts":"1691315700278"} 2023-08-06 09:55:00,279 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-08-06 09:55:00,281 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-08-06 09:55:00,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 360 msec 2023-08-06 09:55:00,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:00,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,430 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:00,430 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,431 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-06 09:55:00,431 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-06 09:55:00,431 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,431 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,432 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,432 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,432 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,432 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,432 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,433 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,434 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,434 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,434 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,434 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,434 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:00,434 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,434 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. started... 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,435 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0584e08f14357669d56c7fddc2d3b3c 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-06 09:55:00,436 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,439 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,439 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,439 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,439 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,439 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,444 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/f7d424c69f024020914e815b9d55700c 2023-08-06 09:55:00,455 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/98a0829f085d4c8ea98ae5814e6f2b73 2023-08-06 09:55:00,459 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/f7d424c69f024020914e815b9d55700c as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/f7d424c69f024020914e815b9d55700c 2023-08-06 09:55:00,463 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/f7d424c69f024020914e815b9d55700c, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:55:00,464 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/98a0829f085d4c8ea98ae5814e6f2b73 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/98a0829f085d4c8ea98ae5814e6f2b73 2023-08-06 09:55:00,467 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/98a0829f085d4c8ea98ae5814e6f2b73, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:55:00,468 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c0584e08f14357669d56c7fddc2d3b3c in 33ms, sequenceid=6, compaction requested=false 2023-08-06 09:55:00,468 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:00,469 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0584e08f14357669d56c7fddc2d3b3c: 2023-08-06 09:55:00,469 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:00,469 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:00,469 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:00,469 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,469 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,469 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,470 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,471 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,471 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,471 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:00,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:00,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-06 09:55:00,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37797a84[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:00,473 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-06 09:55:00,473 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,474 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,474 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,474 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,474 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,474 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,474 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,474 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:00,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,474 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:00,474 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,474 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,474 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:00,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,474 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,474 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,474 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:00,475 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,475 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,475 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,476 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,477 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,477 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,477 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,477 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,477 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,477 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:00,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:00,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:55:00,480 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:55:00,480 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-06 09:55:00,480 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:00,480 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:00,480 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,480 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:00,481 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,481 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,481 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,482 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,481 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,481 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,481 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,482 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,482 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:00,483 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,484 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:00,485 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,485 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,491 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:00,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:00,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,491 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,492 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:00,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,492 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,492 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:00,492 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,492 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,592 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:00,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:00,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:00,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:55:00,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,605 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:00,605 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,605 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-06 09:55:00,605 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-06 09:55:00,606 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,606 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,608 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,608 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,608 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,608 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,608 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,608 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,609 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:00,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:00,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,610 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,610 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. started... 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,610 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,611 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0584e08f14357669d56c7fddc2d3b3c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:55:00,612 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,612 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,612 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,612 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,612 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,613 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,620 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/93c6020b32124bd89a61791418c35a6e 2023-08-06 09:55:00,630 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/942498aa6a2f4277a410039c9c35e218 2023-08-06 09:55:00,634 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/93c6020b32124bd89a61791418c35a6e as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/93c6020b32124bd89a61791418c35a6e 2023-08-06 09:55:00,637 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/93c6020b32124bd89a61791418c35a6e, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:55:00,637 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/942498aa6a2f4277a410039c9c35e218 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/942498aa6a2f4277a410039c9c35e218 2023-08-06 09:55:00,640 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/942498aa6a2f4277a410039c9c35e218, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:55:00,641 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c0584e08f14357669d56c7fddc2d3b3c in 30ms, sequenceid=19, compaction requested=false 2023-08-06 09:55:00,641 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0584e08f14357669d56c7fddc2d3b3c: 2023-08-06 09:55:00,641 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:00,641 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:00,641 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:00,641 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,641 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,641 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,643 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,643 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,643 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,643 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:00,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-06 09:55:00,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69f35ccd[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:00,645 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-06 09:55:00,645 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,647 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,647 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,647 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,648 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:00,647 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,648 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,648 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,648 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,648 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:00,648 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,648 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:00,648 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,648 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,648 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:00,649 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,649 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,649 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,650 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,650 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,650 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,650 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,650 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,650 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,650 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:00,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:00,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:55:00,653 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:55:00,653 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-06 09:55:00,653 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:00,653 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:00,653 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,653 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:00,654 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,654 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,654 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,654 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,654 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,654 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,654 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,654 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,655 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,656 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,657 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:00,657 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,657 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,661 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,661 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,661 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,661 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,661 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,661 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,661 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,661 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:00,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:00,661 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,661 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,661 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:00,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,661 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,662 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:00,662 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,662 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,662 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,662 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,662 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,662 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,662 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,762 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:00,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:00,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:00,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:55:00,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,776 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:00,776 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,776 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-06 09:55:00,776 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-06 09:55:00,776 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,776 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,777 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,777 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,777 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,777 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,778 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,778 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:00,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,779 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,779 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,779 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,779 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:00,779 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,779 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,779 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,779 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,780 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:00,780 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,780 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:00,780 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,780 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,780 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. started... 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,780 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0584e08f14357669d56c7fddc2d3b3c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:55:00,782 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,782 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,782 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,782 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,782 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,782 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,792 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/b421a90bf0e2475c89ce6a9c5befd8a1 2023-08-06 09:55:00,803 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/53073561f0db4d13ba6173ae8684ecee 2023-08-06 09:55:00,806 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/b421a90bf0e2475c89ce6a9c5befd8a1 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/b421a90bf0e2475c89ce6a9c5befd8a1 2023-08-06 09:55:00,809 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/b421a90bf0e2475c89ce6a9c5befd8a1, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:55:00,810 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/53073561f0db4d13ba6173ae8684ecee as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/53073561f0db4d13ba6173ae8684ecee 2023-08-06 09:55:00,812 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/53073561f0db4d13ba6173ae8684ecee, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:55:00,813 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c0584e08f14357669d56c7fddc2d3b3c in 33ms, sequenceid=32, compaction requested=true 2023-08-06 09:55:00,813 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0584e08f14357669d56c7fddc2d3b3c: 2023-08-06 09:55:00,813 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:00,813 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:00,813 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:00,814 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,814 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,814 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,815 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,815 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,815 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,815 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:00,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:00,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-06 09:55:00,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e7ed163[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:00,817 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-06 09:55:00,818 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,820 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,820 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,820 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,820 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,820 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:00,820 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,820 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,820 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:00,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,820 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,820 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,820 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,820 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,820 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:00,820 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:00,825 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,825 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,825 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,826 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,826 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,826 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,826 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,827 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,827 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,828 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:00,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:55:00,830 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:55:00,830 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-06 09:55:00,830 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:00,830 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:00,830 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,830 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:00,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,831 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,832 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,832 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,832 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,833 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:00,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,834 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:00,834 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,835 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,838 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,838 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,837 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,838 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:00,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:00,838 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,838 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,838 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,838 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,838 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,838 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:00,838 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:00,838 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,839 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,839 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,939 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:00,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:00,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:00,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:55:00,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,951 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:00,951 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,951 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-06 09:55:00,951 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-06 09:55:00,951 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,951 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,952 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,952 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,952 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,952 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:00,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:00,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:00,953 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,953 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:00,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,954 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,954 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:00,954 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,954 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,954 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,954 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,954 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:00,954 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,954 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,955 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,955 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:00,955 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:00,954 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,955 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:00,955 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,955 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,955 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,955 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:00,955 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:00,955 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:00,955 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. started... 2023-08-06 09:55:00,955 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0584e08f14357669d56c7fddc2d3b3c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:55:00,957 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,957 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,957 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,957 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,957 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,957 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,963 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/584bfeefee5c419ba93f36c104c9bfdc 2023-08-06 09:55:00,974 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/3d0dc15e3c814f70a1988b7ded763873 2023-08-06 09:55:00,978 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/584bfeefee5c419ba93f36c104c9bfdc as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/584bfeefee5c419ba93f36c104c9bfdc 2023-08-06 09:55:00,981 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/584bfeefee5c419ba93f36c104c9bfdc, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:55:00,982 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/3d0dc15e3c814f70a1988b7ded763873 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/3d0dc15e3c814f70a1988b7ded763873 2023-08-06 09:55:00,985 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/3d0dc15e3c814f70a1988b7ded763873, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:55:00,986 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c0584e08f14357669d56c7fddc2d3b3c in 31ms, sequenceid=45, compaction requested=true 2023-08-06 09:55:00,986 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0584e08f14357669d56c7fddc2d3b3c: 2023-08-06 09:55:00,986 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:00,986 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:00,986 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:00,986 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,986 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:00,986 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:00,987 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,987 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,988 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,988 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:00,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:00,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:00,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-06 09:55:00,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a0c9883[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:00,989 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-06 09:55:00,989 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,990 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,990 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,991 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,991 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,991 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:00,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,991 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,991 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,991 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,991 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:00,991 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,991 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:00,991 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:00,991 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:00,991 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:00,992 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,992 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,992 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,992 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,992 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,992 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,993 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,994 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:00,994 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:00,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,994 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:00,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:00,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:00,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:00,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:55:00,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:55:00,997 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-06 09:55:00,997 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:00,997 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:00,997 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,997 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:00,998 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,998 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,998 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,998 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:00,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:00,998 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,998 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,999 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,998 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:00,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,999 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:00,999 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:01,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,001 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:01,001 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,002 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,006 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,006 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,006 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,007 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:01,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:01,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:01,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,007 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:01,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,007 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:01,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,007 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,008 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,008 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,008 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,008 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,008 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,108 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:01,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:01,120 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:01,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:55:01,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,121 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:01,121 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:01,121 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-06 09:55:01,121 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-06 09:55:01,122 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,122 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,124 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,124 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,124 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,124 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,125 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,125 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:01,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:01,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:01,125 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:01,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,126 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:01,126 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,126 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:01,126 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:01,126 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:01,126 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,126 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:01,126 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:01,126 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:01,126 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:01,126 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:01,126 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,126 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:01,126 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:01,126 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:01,126 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:01,127 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:01,127 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:01,126 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:01,127 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:01,127 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:01,127 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:01,127 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. started... 2023-08-06 09:55:01,127 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0584e08f14357669d56c7fddc2d3b3c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:55:01,128 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,129 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,129 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,129 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:01,129 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,129 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:01,135 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/923543eadc004fa38a164d6e220056f1 2023-08-06 09:55:01,145 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/fdc730624b9d4c93a9ce8ec5b615ac7e 2023-08-06 09:55:01,149 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/923543eadc004fa38a164d6e220056f1 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/923543eadc004fa38a164d6e220056f1 2023-08-06 09:55:01,152 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/923543eadc004fa38a164d6e220056f1, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:55:01,153 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/fdc730624b9d4c93a9ce8ec5b615ac7e as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/fdc730624b9d4c93a9ce8ec5b615ac7e 2023-08-06 09:55:01,155 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/fdc730624b9d4c93a9ce8ec5b615ac7e, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:55:01,156 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c0584e08f14357669d56c7fddc2d3b3c in 29ms, sequenceid=58, compaction requested=true 2023-08-06 09:55:01,156 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0584e08f14357669d56c7fddc2d3b3c: 2023-08-06 09:55:01,156 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:01,156 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:01,156 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:01,156 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,156 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:01,157 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:01,158 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,158 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:01,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:01,158 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,158 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:01,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:01,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:01,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:01,160 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-06 09:55:01,160 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@365d761c[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:01,160 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-06 09:55:01,160 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,162 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,162 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,162 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,162 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:01,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,163 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,163 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:01,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,163 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:01,163 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:01,163 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:01,162 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:01,163 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:01,163 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:01,163 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:01,163 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:01,164 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:01,164 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:01,164 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:01,165 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:01,165 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,165 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:01,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:01,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:01,165 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:01,165 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:01,165 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:01,165 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:01,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:01,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:01,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:55:01,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:55:01,168 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-06 09:55:01,168 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:01,168 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:01,168 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,168 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:01,170 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,170 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,170 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,170 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:01,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:01,170 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,170 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,170 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,170 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:01,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,171 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,171 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:01,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,173 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:01,173 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,173 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,177 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,177 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,177 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,177 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,177 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,177 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,177 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:01,177 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,177 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,177 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:01,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:01,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,177 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,178 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,178 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,178 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,178 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,178 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,178 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,178 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:01,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,178 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,178 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:01,278 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:01,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:01,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:01,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-06 09:55:01,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,290 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:01,290 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:01,291 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-06 09:55:01,291 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-06 09:55:01,291 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,291 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,292 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,292 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,292 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,292 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,293 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,293 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,294 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:01,294 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,294 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:01,294 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-06 09:55:01,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:01,295 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:01,295 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. started... 2023-08-06 09:55:01,295 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0584e08f14357669d56c7fddc2d3b3c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:55:01,295 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,296 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,296 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,296 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:01,297 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,297 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:01,305 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/72c6d814cee5471ebd1ce8388cc3e20e 2023-08-06 09:55:01,316 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/680b7ca6930d4a15815d20db45db4173 2023-08-06 09:55:01,319 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/72c6d814cee5471ebd1ce8388cc3e20e as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/72c6d814cee5471ebd1ce8388cc3e20e 2023-08-06 09:55:01,322 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/72c6d814cee5471ebd1ce8388cc3e20e, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:55:01,323 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/680b7ca6930d4a15815d20db45db4173 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/680b7ca6930d4a15815d20db45db4173 2023-08-06 09:55:01,326 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/680b7ca6930d4a15815d20db45db4173, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:55:01,327 INFO [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c0584e08f14357669d56c7fddc2d3b3c in 32ms, sequenceid=71, compaction requested=true 2023-08-06 09:55:01,327 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0584e08f14357669d56c7fddc2d3b3c: 2023-08-06 09:55:01,327 DEBUG [rs(jenkins-hbase4.apache.org,39585,1691315596578)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:01,327 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:01,327 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:01,327 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,327 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-06 09:55:01,327 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-06 09:55:01,330 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,330 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:01,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:01,330 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,330 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:01,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:01,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:01,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:01,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-06 09:55:01,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45bb512e[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:01,332 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-06 09:55:01,332 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,335 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,335 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,336 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,336 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:01,336 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:01,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,336 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:01,336 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:01,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,336 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:01,336 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:01,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,336 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:01,336 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-06 09:55:01,336 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-06 09:55:01,336 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:01,337 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:01,337 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:01,337 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:01,339 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:01,339 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:01,339 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:01,340 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,340 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-06 09:55:01,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:01,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:01,340 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:01,340 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-06 09:55:01,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:01,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:01,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:01,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39585,1691315596578': 2023-08-06 09:55:01,343 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-06 09:55:01,343 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:01,343 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:01,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-06 09:55:01,343 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,343 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:01,345 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,345 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,345 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,345 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:01,345 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,345 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,345 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,345 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,346 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,346 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:01,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,348 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:01,348 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,348 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-06 09:55:01,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,351 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:01,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:01,351 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:01,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,352 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:01,352 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,352 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:01,352 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,352 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-06 09:55:01,352 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:01,452 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:01,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:01,460 DEBUG [Listener at localhost/36789] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,461 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=c0584e08f14357669d56c7fddc2d3b3c, daughterA=46d65b5a6c59cc9cef60e3eb1ebb9ae7, daughterB=dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:01,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=c0584e08f14357669d56c7fddc2d3b3c, daughterA=46d65b5a6c59cc9cef60e3eb1ebb9ae7, daughterB=dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:01,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=c0584e08f14357669d56c7fddc2d3b3c, daughterA=46d65b5a6c59cc9cef60e3eb1ebb9ae7, daughterB=dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:01,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=c0584e08f14357669d56c7fddc2d3b3c, daughterA=46d65b5a6c59cc9cef60e3eb1ebb9ae7, daughterB=dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:01,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c0584e08f14357669d56c7fddc2d3b3c, UNASSIGN}] 2023-08-06 09:55:01,465 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c0584e08f14357669d56c7fddc2d3b3c, UNASSIGN 2023-08-06 09:55:01,465 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=c0584e08f14357669d56c7fddc2d3b3c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,465 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315701465"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315701465"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315701465"}]},"ts":"1691315701465"} 2023-08-06 09:55:01,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure c0584e08f14357669d56c7fddc2d3b3c, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:55:01,618 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c0584e08f14357669d56c7fddc2d3b3c, disabling compactions & flushes 2023-08-06 09:55:01,618 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:01,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:01,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. after waiting 0 ms 2023-08-06 09:55:01,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:01,619 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing c0584e08f14357669d56c7fddc2d3b3c 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-06 09:55:01,626 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/0d90af5e5e9b44acbe66f51331d3fd46 2023-08-06 09:55:01,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/c8517e04522a4021a537713ff0d99fd4 2023-08-06 09:55:01,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf1/0d90af5e5e9b44acbe66f51331d3fd46 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/0d90af5e5e9b44acbe66f51331d3fd46 2023-08-06 09:55:01,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/0d90af5e5e9b44acbe66f51331d3fd46, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:55:01,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.tmp/cf2/c8517e04522a4021a537713ff0d99fd4 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/c8517e04522a4021a537713ff0d99fd4 2023-08-06 09:55:01,649 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/c8517e04522a4021a537713ff0d99fd4, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:55:01,649 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c0584e08f14357669d56c7fddc2d3b3c in 31ms, sequenceid=82, compaction requested=true 2023-08-06 09:55:01,654 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-06 09:55:01,654 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:55:01,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. 2023-08-06 09:55:01,654 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c0584e08f14357669d56c7fddc2d3b3c: 2023-08-06 09:55:01,656 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,656 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=c0584e08f14357669d56c7fddc2d3b3c, regionState=CLOSED 2023-08-06 09:55:01,656 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315701656"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315701656"}]},"ts":"1691315701656"} 2023-08-06 09:55:01,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-08-06 09:55:01,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure c0584e08f14357669d56c7fddc2d3b3c, server=jenkins-hbase4.apache.org,39585,1691315596578 in 191 msec 2023-08-06 09:55:01,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-08-06 09:55:01,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c0584e08f14357669d56c7fddc2d3b3c, UNASSIGN in 201 msec 2023-08-06 09:55:01,677 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=c0584e08f14357669d56c7fddc2d3b3c, threads=14 2023-08-06 09:55:01,677 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/3d0dc15e3c814f70a1988b7ded763873 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,677 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/53073561f0db4d13ba6173ae8684ecee for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,677 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/942498aa6a2f4277a410039c9c35e218 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,677 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/680b7ca6930d4a15815d20db45db4173 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,678 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/98a0829f085d4c8ea98ae5814e6f2b73 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,678 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/c8517e04522a4021a537713ff0d99fd4 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,678 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/fdc730624b9d4c93a9ce8ec5b615ac7e for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,678 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/0d90af5e5e9b44acbe66f51331d3fd46 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,682 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/584bfeefee5c419ba93f36c104c9bfdc for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,683 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/72c6d814cee5471ebd1ce8388cc3e20e for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,683 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/923543eadc004fa38a164d6e220056f1 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,684 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/93c6020b32124bd89a61791418c35a6e for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,684 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/b421a90bf0e2475c89ce6a9c5befd8a1 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,684 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/f7d424c69f024020914e815b9d55700c for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,688 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/f7d424c69f024020914e815b9d55700c, top=false 2023-08-06 09:55:01,689 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/98a0829f085d4c8ea98ae5814e6f2b73, top=false 2023-08-06 09:55:01,689 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/3d0dc15e3c814f70a1988b7ded763873, top=false 2023-08-06 09:55:01,691 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/680b7ca6930d4a15815d20db45db4173, top=false 2023-08-06 09:55:01,692 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/923543eadc004fa38a164d6e220056f1, top=false 2023-08-06 09:55:01,692 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/fdc730624b9d4c93a9ce8ec5b615ac7e, top=false 2023-08-06 09:55:01,694 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/53073561f0db4d13ba6173ae8684ecee, top=false 2023-08-06 09:55:01,695 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/c8517e04522a4021a537713ff0d99fd4, top=false 2023-08-06 09:55:01,695 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/93c6020b32124bd89a61791418c35a6e, top=false 2023-08-06 09:55:01,695 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/72c6d814cee5471ebd1ce8388cc3e20e, top=false 2023-08-06 09:55:01,696 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/584bfeefee5c419ba93f36c104c9bfdc, top=false 2023-08-06 09:55:01,698 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/942498aa6a2f4277a410039c9c35e218, top=false 2023-08-06 09:55:01,698 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/0d90af5e5e9b44acbe66f51331d3fd46, top=false 2023-08-06 09:55:01,707 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/b421a90bf0e2475c89ce6a9c5befd8a1, top=false 2023-08-06 09:55:01,715 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-98a0829f085d4c8ea98ae5814e6f2b73 for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,721 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/98a0829f085d4c8ea98ae5814e6f2b73 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,731 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-b421a90bf0e2475c89ce6a9c5befd8a1 for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,731 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-0d90af5e5e9b44acbe66f51331d3fd46 for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,731 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-f7d424c69f024020914e815b9d55700c for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,734 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/0d90af5e5e9b44acbe66f51331d3fd46 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,734 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/b421a90bf0e2475c89ce6a9c5befd8a1 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,734 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/f7d424c69f024020914e815b9d55700c for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,739 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-584bfeefee5c419ba93f36c104c9bfdc for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,739 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-680b7ca6930d4a15815d20db45db4173 for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,739 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-72c6d814cee5471ebd1ce8388cc3e20e for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,739 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-53073561f0db4d13ba6173ae8684ecee for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,739 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-923543eadc004fa38a164d6e220056f1 for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,740 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-93c6020b32124bd89a61791418c35a6e for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,740 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-3d0dc15e3c814f70a1988b7ded763873 for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,748 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-942498aa6a2f4277a410039c9c35e218 for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,749 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/680b7ca6930d4a15815d20db45db4173 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,750 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-c8517e04522a4021a537713ff0d99fd4 for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,750 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/93c6020b32124bd89a61791418c35a6e for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,751 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/72c6d814cee5471ebd1ce8388cc3e20e for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,752 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/923543eadc004fa38a164d6e220056f1 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,752 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/53073561f0db4d13ba6173ae8684ecee for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,753 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/584bfeefee5c419ba93f36c104c9bfdc for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,754 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/.splits/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-fdc730624b9d4c93a9ce8ec5b615ac7e for child: 46d65b5a6c59cc9cef60e3eb1ebb9ae7, parent: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,755 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/942498aa6a2f4277a410039c9c35e218 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,755 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/c8517e04522a4021a537713ff0d99fd4 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,757 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/fdc730624b9d4c93a9ce8ec5b615ac7e for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,758 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/3d0dc15e3c814f70a1988b7ded763873 for region: c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:01,758 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region c0584e08f14357669d56c7fddc2d3b3c Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-08-06 09:55:01,796 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:55:01,798 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:55:01,800 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1691315701800"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1691315701800"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1691315701800"}]},"ts":"1691315701800"} 2023-08-06 09:55:01,800 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691315701800"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315701800"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315701800"}]},"ts":"1691315701800"} 2023-08-06 09:55:01,800 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691315701800"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315701800"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315701800"}]},"ts":"1691315701800"} 2023-08-06 09:55:01,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=46d65b5a6c59cc9cef60e3eb1ebb9ae7, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=dc8beded904afdf12cba9cbc2cea7f75, ASSIGN}] 2023-08-06 09:55:01,804 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=dc8beded904afdf12cba9cbc2cea7f75, ASSIGN 2023-08-06 09:55:01,805 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=dc8beded904afdf12cba9cbc2cea7f75, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39585,1691315596578; forceNewPlan=false, retain=false 2023-08-06 09:55:01,806 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=46d65b5a6c59cc9cef60e3eb1ebb9ae7, ASSIGN 2023-08-06 09:55:01,806 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=46d65b5a6c59cc9cef60e3eb1ebb9ae7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39585,1691315596578; forceNewPlan=false, retain=false 2023-08-06 09:55:01,907 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-06 09:55:01,955 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-06 09:55:01,957 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=46d65b5a6c59cc9cef60e3eb1ebb9ae7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,958 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691315701957"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315701957"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315701957"}]},"ts":"1691315701957"} 2023-08-06 09:55:01,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure 46d65b5a6c59cc9cef60e3eb1ebb9ae7, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:55:01,962 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=dc8beded904afdf12cba9cbc2cea7f75, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:01,962 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691315701962"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315701962"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315701962"}]},"ts":"1691315701962"} 2023-08-06 09:55:01,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure dc8beded904afdf12cba9cbc2cea7f75, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:55:02,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7. 2023-08-06 09:55:02,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 46d65b5a6c59cc9cef60e3eb1ebb9ae7, NAME => 'testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.', STARTKEY => '', ENDKEY => '71'} 2023-08-06 09:55:02,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:55:02,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:55:02,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,114 INFO [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,115 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1 2023-08-06 09:55:02,115 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1 2023-08-06 09:55:02,115 INFO [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 46d65b5a6c59cc9cef60e3eb1ebb9ae7 columnFamilyName cf1 2023-08-06 09:55:02,119 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-0d90af5e5e9b44acbe66f51331d3fd46 2023-08-06 09:55:02,122 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-584bfeefee5c419ba93f36c104c9bfdc 2023-08-06 09:55:02,124 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-72c6d814cee5471ebd1ce8388cc3e20e 2023-08-06 09:55:02,126 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-923543eadc004fa38a164d6e220056f1 2023-08-06 09:55:02,129 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-93c6020b32124bd89a61791418c35a6e 2023-08-06 09:55:02,131 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-b421a90bf0e2475c89ce6a9c5befd8a1 2023-08-06 09:55:02,134 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-f7d424c69f024020914e815b9d55700c 2023-08-06 09:55:02,134 INFO [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(310): Store=46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:02,134 INFO [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,135 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2 2023-08-06 09:55:02,135 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2 2023-08-06 09:55:02,135 INFO [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 46d65b5a6c59cc9cef60e3eb1ebb9ae7 columnFamilyName cf2 2023-08-06 09:55:02,139 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-3d0dc15e3c814f70a1988b7ded763873 2023-08-06 09:55:02,142 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-53073561f0db4d13ba6173ae8684ecee 2023-08-06 09:55:02,144 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-680b7ca6930d4a15815d20db45db4173 2023-08-06 09:55:02,147 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-942498aa6a2f4277a410039c9c35e218 2023-08-06 09:55:02,149 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-98a0829f085d4c8ea98ae5814e6f2b73 2023-08-06 09:55:02,152 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-c8517e04522a4021a537713ff0d99fd4 2023-08-06 09:55:02,154 DEBUG [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-fdc730624b9d4c93a9ce8ec5b615ac7e 2023-08-06 09:55:02,154 INFO [StoreOpener-46d65b5a6c59cc9cef60e3eb1ebb9ae7-1] regionserver.HStore(310): Store=46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:02,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:55:02,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 46d65b5a6c59cc9cef60e3eb1ebb9ae7; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69967181, jitterRate=0.04259224236011505}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:55:02,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 46d65b5a6c59cc9cef60e3eb1ebb9ae7: 2023-08-06 09:55:02,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7., pid=133, masterSystemTime=1691315702110 2023-08-06 09:55:02,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:55:02,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-06 09:55:02,161 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:55:02,161 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:55:02,161 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-06 09:55:02,161 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-06 09:55:02,161 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] regionserver.HStore(1912): 46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1 is initiating minor compaction (all files) 2023-08-06 09:55:02,161 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] regionserver.HStore(1912): 46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2 is initiating minor compaction (all files) 2023-08-06 09:55:02,161 INFO [RS:0;jenkins-hbase4:39585-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1 in testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7. 2023-08-06 09:55:02,161 INFO [RS:0;jenkins-hbase4:39585-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2 in testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7. 2023-08-06 09:55:02,161 INFO [RS:0;jenkins-hbase4:39585-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-f7d424c69f024020914e815b9d55700c, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-93c6020b32124bd89a61791418c35a6e, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-b421a90bf0e2475c89ce6a9c5befd8a1, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-584bfeefee5c419ba93f36c104c9bfdc, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-923543eadc004fa38a164d6e220056f1, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-72c6d814cee5471ebd1ce8388cc3e20e, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-0d90af5e5e9b44acbe66f51331d3fd46] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/.tmp, totalSize=34.9 K 2023-08-06 09:55:02,161 INFO [RS:0;jenkins-hbase4:39585-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-98a0829f085d4c8ea98ae5814e6f2b73, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-942498aa6a2f4277a410039c9c35e218, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-53073561f0db4d13ba6173ae8684ecee, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-3d0dc15e3c814f70a1988b7ded763873, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-fdc730624b9d4c93a9ce8ec5b615ac7e, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-680b7ca6930d4a15815d20db45db4173, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-c8517e04522a4021a537713ff0d99fd4] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/.tmp, totalSize=34.9 K 2023-08-06 09:55:02,162 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-f7d424c69f024020914e815b9d55700c, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315700425 2023-08-06 09:55:02,162 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-93c6020b32124bd89a61791418c35a6e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315700594 2023-08-06 09:55:02,162 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-98a0829f085d4c8ea98ae5814e6f2b73, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315700425 2023-08-06 09:55:02,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7. 2023-08-06 09:55:02,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7. 2023-08-06 09:55:02,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75. 2023-08-06 09:55:02,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dc8beded904afdf12cba9cbc2cea7f75, NAME => 'testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75.', STARTKEY => '71', ENDKEY => ''} 2023-08-06 09:55:02,162 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=46d65b5a6c59cc9cef60e3eb1ebb9ae7, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:02,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:55:02,163 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691315702162"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315702162"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315702162"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315702162"}]},"ts":"1691315702162"} 2023-08-06 09:55:02,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:55:02,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,163 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-b421a90bf0e2475c89ce6a9c5befd8a1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315700763 2023-08-06 09:55:02,164 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-584bfeefee5c419ba93f36c104c9bfdc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315700940 2023-08-06 09:55:02,164 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-942498aa6a2f4277a410039c9c35e218, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315700594 2023-08-06 09:55:02,164 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-923543eadc004fa38a164d6e220056f1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315701109 2023-08-06 09:55:02,165 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-72c6d814cee5471ebd1ce8388cc3e20e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315701280 2023-08-06 09:55:02,165 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-53073561f0db4d13ba6173ae8684ecee, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315700763 2023-08-06 09:55:02,165 INFO [StoreOpener-dc8beded904afdf12cba9cbc2cea7f75-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,165 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-0d90af5e5e9b44acbe66f51331d3fd46, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315701454 2023-08-06 09:55:02,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-08-06 09:55:02,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure 46d65b5a6c59cc9cef60e3eb1ebb9ae7, server=jenkins-hbase4.apache.org,39585,1691315596578 in 205 msec 2023-08-06 09:55:02,166 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-3d0dc15e3c814f70a1988b7ded763873, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315700940 2023-08-06 09:55:02,166 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-fdc730624b9d4c93a9ce8ec5b615ac7e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315701109 2023-08-06 09:55:02,167 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-680b7ca6930d4a15815d20db45db4173, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315701280 2023-08-06 09:55:02,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=46d65b5a6c59cc9cef60e3eb1ebb9ae7, ASSIGN in 363 msec 2023-08-06 09:55:02,169 DEBUG [StoreOpener-dc8beded904afdf12cba9cbc2cea7f75-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75/cf1 2023-08-06 09:55:02,169 DEBUG [StoreOpener-dc8beded904afdf12cba9cbc2cea7f75-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75/cf1 2023-08-06 09:55:02,169 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-c8517e04522a4021a537713ff0d99fd4, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315701454 2023-08-06 09:55:02,169 INFO [StoreOpener-dc8beded904afdf12cba9cbc2cea7f75-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dc8beded904afdf12cba9cbc2cea7f75 columnFamilyName cf1 2023-08-06 09:55:02,170 INFO [StoreOpener-dc8beded904afdf12cba9cbc2cea7f75-1] regionserver.HStore(310): Store=dc8beded904afdf12cba9cbc2cea7f75/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:02,170 INFO [StoreOpener-dc8beded904afdf12cba9cbc2cea7f75-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,171 DEBUG [StoreOpener-dc8beded904afdf12cba9cbc2cea7f75-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75/cf2 2023-08-06 09:55:02,171 DEBUG [StoreOpener-dc8beded904afdf12cba9cbc2cea7f75-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75/cf2 2023-08-06 09:55:02,173 INFO [StoreOpener-dc8beded904afdf12cba9cbc2cea7f75-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dc8beded904afdf12cba9cbc2cea7f75 columnFamilyName cf2 2023-08-06 09:55:02,173 INFO [StoreOpener-dc8beded904afdf12cba9cbc2cea7f75-1] regionserver.HStore(310): Store=dc8beded904afdf12cba9cbc2cea7f75/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:02,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:55:02,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dc8beded904afdf12cba9cbc2cea7f75; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74409264, jitterRate=0.10878443717956543}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:55:02,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dc8beded904afdf12cba9cbc2cea7f75: 2023-08-06 09:55:02,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75., pid=134, masterSystemTime=1691315702110 2023-08-06 09:55:02,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75. 2023-08-06 09:55:02,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75. 2023-08-06 09:55:02,182 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=dc8beded904afdf12cba9cbc2cea7f75, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:02,182 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691315702182"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315702182"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315702182"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315702182"}]},"ts":"1691315702182"} 2023-08-06 09:55:02,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-08-06 09:55:02,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure dc8beded904afdf12cba9cbc2cea7f75, server=jenkins-hbase4.apache.org,39585,1691315596578 in 220 msec 2023-08-06 09:55:02,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-08-06 09:55:02,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=dc8beded904afdf12cba9cbc2cea7f75, ASSIGN in 382 msec 2023-08-06 09:55:02,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=c0584e08f14357669d56c7fddc2d3b3c, daughterA=46d65b5a6c59cc9cef60e3eb1ebb9ae7, daughterB=dc8beded904afdf12cba9cbc2cea7f75 in 725 msec 2023-08-06 09:55:02,194 INFO [RS:0;jenkins-hbase4:39585-shortCompactions-0] throttle.PressureAwareThroughputController(145): 46d65b5a6c59cc9cef60e3eb1ebb9ae7#cf1#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:55:02,194 INFO [RS:0;jenkins-hbase4:39585-longCompactions-0] throttle.PressureAwareThroughputController(145): 46d65b5a6c59cc9cef60e3eb1ebb9ae7#cf2#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:55:02,211 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/.tmp/cf2/fc25a1b6cd7248838d5c5743062e044b as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/fc25a1b6cd7248838d5c5743062e044b 2023-08-06 09:55:02,211 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/.tmp/cf1/7f05c7883b3c4e608de5cc6ce0a3899e as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/7f05c7883b3c4e608de5cc6ce0a3899e 2023-08-06 09:55:02,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39585] ipc.CallRunner(144): callId: 993 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:45382 deadline: 1691315762212, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c. is not online on jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:02,217 INFO [RS:0;jenkins-hbase4:39585-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2 of 46d65b5a6c59cc9cef60e3eb1ebb9ae7 into fc25a1b6cd7248838d5c5743062e044b(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:55:02,217 INFO [RS:0;jenkins-hbase4:39585-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1 of 46d65b5a6c59cc9cef60e3eb1ebb9ae7 into 7f05c7883b3c4e608de5cc6ce0a3899e(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:55:02,217 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 46d65b5a6c59cc9cef60e3eb1ebb9ae7: 2023-08-06 09:55:02,217 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 46d65b5a6c59cc9cef60e3eb1ebb9ae7: 2023-08-06 09:55:02,217 INFO [RS:0;jenkins-hbase4:39585-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7., storeName=46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2, priority=9, startTime=1691315702159; duration=0sec 2023-08-06 09:55:02,217 INFO [RS:0;jenkins-hbase4:39585-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7., storeName=46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1, priority=9, startTime=1691315702159; duration=0sec 2023-08-06 09:55:02,218 DEBUG [RS:0;jenkins-hbase4:39585-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:55:02,218 DEBUG [RS:0;jenkins-hbase4:39585-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:55:02,324 DEBUG [Listener at localhost/36789] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-08-06 09:55:02,326 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:55:02,328 INFO [Listener at localhost/36789] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-08-06 09:55:02,328 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-08-06 09:55:02,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-06 09:55:02,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-08-06 09:55:02,331 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315702331"}]},"ts":"1691315702331"} 2023-08-06 09:55:02,332 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-08-06 09:55:02,334 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-08-06 09:55:02,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=46d65b5a6c59cc9cef60e3eb1ebb9ae7, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=dc8beded904afdf12cba9cbc2cea7f75, UNASSIGN}] 2023-08-06 09:55:02,335 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=dc8beded904afdf12cba9cbc2cea7f75, UNASSIGN 2023-08-06 09:55:02,336 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=dc8beded904afdf12cba9cbc2cea7f75, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:02,336 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691315702336"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315702336"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315702336"}]},"ts":"1691315702336"} 2023-08-06 09:55:02,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure dc8beded904afdf12cba9cbc2cea7f75, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:55:02,339 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=46d65b5a6c59cc9cef60e3eb1ebb9ae7, UNASSIGN 2023-08-06 09:55:02,340 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=46d65b5a6c59cc9cef60e3eb1ebb9ae7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:02,340 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691315702340"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315702340"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315702340"}]},"ts":"1691315702340"} 2023-08-06 09:55:02,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 46d65b5a6c59cc9cef60e3eb1ebb9ae7, server=jenkins-hbase4.apache.org,39585,1691315596578}] 2023-08-06 09:55:02,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-08-06 09:55:02,489 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 46d65b5a6c59cc9cef60e3eb1ebb9ae7, disabling compactions & flushes 2023-08-06 09:55:02,489 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7. 2023-08-06 09:55:02,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7. 2023-08-06 09:55:02,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7. after waiting 0 ms 2023-08-06 09:55:02,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7. 2023-08-06 09:55:02,490 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-f7d424c69f024020914e815b9d55700c, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-93c6020b32124bd89a61791418c35a6e, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-b421a90bf0e2475c89ce6a9c5befd8a1, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-584bfeefee5c419ba93f36c104c9bfdc, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-923543eadc004fa38a164d6e220056f1, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-72c6d814cee5471ebd1ce8388cc3e20e, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-0d90af5e5e9b44acbe66f51331d3fd46] to archive 2023-08-06 09:55:02,491 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:55:02,492 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-f7d424c69f024020914e815b9d55700c to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-f7d424c69f024020914e815b9d55700c 2023-08-06 09:55:02,493 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-93c6020b32124bd89a61791418c35a6e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-93c6020b32124bd89a61791418c35a6e 2023-08-06 09:55:02,494 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-b421a90bf0e2475c89ce6a9c5befd8a1 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-b421a90bf0e2475c89ce6a9c5befd8a1 2023-08-06 09:55:02,495 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-584bfeefee5c419ba93f36c104c9bfdc to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-584bfeefee5c419ba93f36c104c9bfdc 2023-08-06 09:55:02,496 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-923543eadc004fa38a164d6e220056f1 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-923543eadc004fa38a164d6e220056f1 2023-08-06 09:55:02,497 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-72c6d814cee5471ebd1ce8388cc3e20e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-72c6d814cee5471ebd1ce8388cc3e20e 2023-08-06 09:55:02,498 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-0d90af5e5e9b44acbe66f51331d3fd46 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-0d90af5e5e9b44acbe66f51331d3fd46 2023-08-06 09:55:02,499 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-98a0829f085d4c8ea98ae5814e6f2b73, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-942498aa6a2f4277a410039c9c35e218, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-53073561f0db4d13ba6173ae8684ecee, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-3d0dc15e3c814f70a1988b7ded763873, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-fdc730624b9d4c93a9ce8ec5b615ac7e, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-680b7ca6930d4a15815d20db45db4173, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-c8517e04522a4021a537713ff0d99fd4] to archive 2023-08-06 09:55:02,499 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:55:02,501 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-98a0829f085d4c8ea98ae5814e6f2b73 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-98a0829f085d4c8ea98ae5814e6f2b73 2023-08-06 09:55:02,501 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-942498aa6a2f4277a410039c9c35e218 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-942498aa6a2f4277a410039c9c35e218 2023-08-06 09:55:02,502 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-53073561f0db4d13ba6173ae8684ecee to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-53073561f0db4d13ba6173ae8684ecee 2023-08-06 09:55:02,503 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-3d0dc15e3c814f70a1988b7ded763873 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-3d0dc15e3c814f70a1988b7ded763873 2023-08-06 09:55:02,504 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-fdc730624b9d4c93a9ce8ec5b615ac7e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-fdc730624b9d4c93a9ce8ec5b615ac7e 2023-08-06 09:55:02,505 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-680b7ca6930d4a15815d20db45db4173 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-680b7ca6930d4a15815d20db45db4173 2023-08-06 09:55:02,506 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-c8517e04522a4021a537713ff0d99fd4 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/testSplitTableRegionEmptyDaughter=c0584e08f14357669d56c7fddc2d3b3c-c8517e04522a4021a537713ff0d99fd4 2023-08-06 09:55:02,509 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-06 09:55:02,509 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:55:02,509 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7. 2023-08-06 09:55:02,509 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 46d65b5a6c59cc9cef60e3eb1ebb9ae7: 2023-08-06 09:55:02,510 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dc8beded904afdf12cba9cbc2cea7f75, disabling compactions & flushes 2023-08-06 09:55:02,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75. 2023-08-06 09:55:02,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75. 2023-08-06 09:55:02,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75. after waiting 0 ms 2023-08-06 09:55:02,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75. 2023-08-06 09:55:02,512 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=46d65b5a6c59cc9cef60e3eb1ebb9ae7, regionState=CLOSED 2023-08-06 09:55:02,512 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691315702512"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315702512"}]},"ts":"1691315702512"} 2023-08-06 09:55:02,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-08-06 09:55:02,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-08-06 09:55:02,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 46d65b5a6c59cc9cef60e3eb1ebb9ae7, server=jenkins-hbase4.apache.org,39585,1691315596578 in 172 msec 2023-08-06 09:55:02,515 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:55:02,515 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75. 2023-08-06 09:55:02,515 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dc8beded904afdf12cba9cbc2cea7f75: 2023-08-06 09:55:02,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=46d65b5a6c59cc9cef60e3eb1ebb9ae7, UNASSIGN in 179 msec 2023-08-06 09:55:02,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,516 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=dc8beded904afdf12cba9cbc2cea7f75, regionState=CLOSED 2023-08-06 09:55:02,516 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1691315702516"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315702516"}]},"ts":"1691315702516"} 2023-08-06 09:55:02,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-08-06 09:55:02,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure dc8beded904afdf12cba9cbc2cea7f75, server=jenkins-hbase4.apache.org,39585,1691315596578 in 180 msec 2023-08-06 09:55:02,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-08-06 09:55:02,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=dc8beded904afdf12cba9cbc2cea7f75, UNASSIGN in 183 msec 2023-08-06 09:55:02,520 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315702520"}]},"ts":"1691315702520"} 2023-08-06 09:55:02,521 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-08-06 09:55:02,522 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-08-06 09:55:02,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 195 msec 2023-08-06 09:55:02,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-08-06 09:55:02,634 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-08-06 09:55:02,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-08-06 09:55:02,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-06 09:55:02,637 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-06 09:55:02,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-08-06 09:55:02,637 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-06 09:55:02,641 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:02,641 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,641 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,643 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75/recovered.edits] 2023-08-06 09:55:02,643 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/recovered.edits] 2023-08-06 09:55:02,643 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/recovered.edits] 2023-08-06 09:55:02,646 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/0d90af5e5e9b44acbe66f51331d3fd46 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/0d90af5e5e9b44acbe66f51331d3fd46 2023-08-06 09:55:02,646 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/7f05c7883b3c4e608de5cc6ce0a3899e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf1/7f05c7883b3c4e608de5cc6ce0a3899e 2023-08-06 09:55:02,648 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/584bfeefee5c419ba93f36c104c9bfdc to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/584bfeefee5c419ba93f36c104c9bfdc 2023-08-06 09:55:02,649 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75/recovered.edits/88.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75/recovered.edits/88.seqid 2023-08-06 09:55:02,649 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/dc8beded904afdf12cba9cbc2cea7f75 2023-08-06 09:55:02,649 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/fc25a1b6cd7248838d5c5743062e044b to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/cf2/fc25a1b6cd7248838d5c5743062e044b 2023-08-06 09:55:02,649 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/72c6d814cee5471ebd1ce8388cc3e20e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/72c6d814cee5471ebd1ce8388cc3e20e 2023-08-06 09:55:02,651 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/923543eadc004fa38a164d6e220056f1 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/923543eadc004fa38a164d6e220056f1 2023-08-06 09:55:02,652 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/recovered.edits/90.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7/recovered.edits/90.seqid 2023-08-06 09:55:02,652 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/93c6020b32124bd89a61791418c35a6e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/93c6020b32124bd89a61791418c35a6e 2023-08-06 09:55:02,652 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/46d65b5a6c59cc9cef60e3eb1ebb9ae7 2023-08-06 09:55:02,653 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/b421a90bf0e2475c89ce6a9c5befd8a1 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/b421a90bf0e2475c89ce6a9c5befd8a1 2023-08-06 09:55:02,654 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/f7d424c69f024020914e815b9d55700c to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf1/f7d424c69f024020914e815b9d55700c 2023-08-06 09:55:02,656 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/3d0dc15e3c814f70a1988b7ded763873 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/3d0dc15e3c814f70a1988b7ded763873 2023-08-06 09:55:02,657 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/53073561f0db4d13ba6173ae8684ecee to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/53073561f0db4d13ba6173ae8684ecee 2023-08-06 09:55:02,658 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/680b7ca6930d4a15815d20db45db4173 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/680b7ca6930d4a15815d20db45db4173 2023-08-06 09:55:02,659 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/942498aa6a2f4277a410039c9c35e218 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/942498aa6a2f4277a410039c9c35e218 2023-08-06 09:55:02,660 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/98a0829f085d4c8ea98ae5814e6f2b73 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/98a0829f085d4c8ea98ae5814e6f2b73 2023-08-06 09:55:02,661 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/c8517e04522a4021a537713ff0d99fd4 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/c8517e04522a4021a537713ff0d99fd4 2023-08-06 09:55:02,662 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/fdc730624b9d4c93a9ce8ec5b615ac7e to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/cf2/fdc730624b9d4c93a9ce8ec5b615ac7e 2023-08-06 09:55:02,664 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/recovered.edits/85.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c/recovered.edits/85.seqid 2023-08-06 09:55:02,664 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegionEmptyDaughter/c0584e08f14357669d56c7fddc2d3b3c 2023-08-06 09:55:02,664 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-08-06 09:55:02,666 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-06 09:55:02,668 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-08-06 09:55:02,669 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-08-06 09:55:02,670 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-06 09:55:02,670 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-08-06 09:55:02,670 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315702670"}]},"ts":"9223372036854775807"} 2023-08-06 09:55:02,670 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315702670"}]},"ts":"9223372036854775807"} 2023-08-06 09:55:02,670 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315702670"}]},"ts":"9223372036854775807"} 2023-08-06 09:55:02,672 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-06 09:55:02,672 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c0584e08f14357669d56c7fddc2d3b3c, NAME => 'testSplitTableRegionEmptyDaughter,,1691315699921.c0584e08f14357669d56c7fddc2d3b3c.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 46d65b5a6c59cc9cef60e3eb1ebb9ae7, NAME => 'testSplitTableRegionEmptyDaughter,,1691315701460.46d65b5a6c59cc9cef60e3eb1ebb9ae7.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => dc8beded904afdf12cba9cbc2cea7f75, NAME => 'testSplitTableRegionEmptyDaughter,71,1691315701460.dc8beded904afdf12cba9cbc2cea7f75.', STARTKEY => '71', ENDKEY => ''}] 2023-08-06 09:55:02,672 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-08-06 09:55:02,672 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691315702672"}]},"ts":"9223372036854775807"} 2023-08-06 09:55:02,673 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-08-06 09:55:02,675 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-06 09:55:02,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 41 msec 2023-08-06 09:55:02,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-08-06 09:55:02,739 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-08-06 09:55:02,753 INFO [Listener at localhost/36789] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=379 (was 378) - Thread LEAK? -, OpenFileDescriptor=835 (was 835), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=151 (was 146) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=2275 (was 2281) 2023-08-06 09:55:02,766 INFO [Listener at localhost/36789] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=379, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=151, ProcessCount=171, AvailableMemoryMB=2275 2023-08-06 09:55:02,766 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:55:02,768 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-06 09:55:02,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-06 09:55:02,770 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-08-06 09:55:02,771 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-08-06 09:55:02,772 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-06 09:55:02,773 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:02,773 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5 empty. 2023-08-06 09:55:02,774 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:02,774 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-08-06 09:55:02,786 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-08-06 09:55:02,787 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => 976d2fa78f57b6e8c06e1e70f4d3ffe5, NAME => 'testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp 2023-08-06 09:55:02,797 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:55:02,797 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing 976d2fa78f57b6e8c06e1e70f4d3ffe5, disabling compactions & flushes 2023-08-06 09:55:02,797 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:02,797 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:02,797 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. after waiting 0 ms 2023-08-06 09:55:02,797 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:02,797 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:02,797 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for 976d2fa78f57b6e8c06e1e70f4d3ffe5: 2023-08-06 09:55:02,799 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-08-06 09:55:02,799 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315702799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315702799"}]},"ts":"1691315702799"} 2023-08-06 09:55:02,801 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-06 09:55:02,801 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-06 09:55:02,801 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315702801"}]},"ts":"1691315702801"} 2023-08-06 09:55:02,802 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-08-06 09:55:02,806 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-06 09:55:02,807 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-06 09:55:02,807 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-06 09:55:02,807 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-06 09:55:02,807 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-06 09:55:02,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=976d2fa78f57b6e8c06e1e70f4d3ffe5, ASSIGN}] 2023-08-06 09:55:02,808 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=976d2fa78f57b6e8c06e1e70f4d3ffe5, ASSIGN 2023-08-06 09:55:02,809 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=976d2fa78f57b6e8c06e1e70f4d3ffe5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40847,1691315596668; forceNewPlan=false, retain=false 2023-08-06 09:55:02,959 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-06 09:55:02,960 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=976d2fa78f57b6e8c06e1e70f4d3ffe5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:02,961 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315702960"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315702960"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315702960"}]},"ts":"1691315702960"} 2023-08-06 09:55:02,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 976d2fa78f57b6e8c06e1e70f4d3ffe5, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:55:03,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:03,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 976d2fa78f57b6e8c06e1e70f4d3ffe5, NAME => 'testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.', STARTKEY => '', ENDKEY => ''} 2023-08-06 09:55:03,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:55:03,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:03,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:55:03,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:03,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:03,121 INFO [StoreOpener-976d2fa78f57b6e8c06e1e70f4d3ffe5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:03,122 DEBUG [StoreOpener-976d2fa78f57b6e8c06e1e70f4d3ffe5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1 2023-08-06 09:55:03,122 DEBUG [StoreOpener-976d2fa78f57b6e8c06e1e70f4d3ffe5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1 2023-08-06 09:55:03,123 INFO [StoreOpener-976d2fa78f57b6e8c06e1e70f4d3ffe5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 976d2fa78f57b6e8c06e1e70f4d3ffe5 columnFamilyName cf1 2023-08-06 09:55:03,123 INFO [StoreOpener-976d2fa78f57b6e8c06e1e70f4d3ffe5-1] regionserver.HStore(310): Store=976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:03,123 INFO [StoreOpener-976d2fa78f57b6e8c06e1e70f4d3ffe5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:03,124 DEBUG [StoreOpener-976d2fa78f57b6e8c06e1e70f4d3ffe5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2 2023-08-06 09:55:03,124 DEBUG [StoreOpener-976d2fa78f57b6e8c06e1e70f4d3ffe5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2 2023-08-06 09:55:03,124 INFO [StoreOpener-976d2fa78f57b6e8c06e1e70f4d3ffe5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 976d2fa78f57b6e8c06e1e70f4d3ffe5 columnFamilyName cf2 2023-08-06 09:55:03,125 INFO [StoreOpener-976d2fa78f57b6e8c06e1e70f4d3ffe5-1] regionserver.HStore(310): Store=976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:03,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:03,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:03,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:55:03,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:03,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-06 09:55:03,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 976d2fa78f57b6e8c06e1e70f4d3ffe5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62215714, jitterRate=-0.07291361689567566}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:55:03,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 976d2fa78f57b6e8c06e1e70f4d3ffe5: 2023-08-06 09:55:03,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5., pid=143, masterSystemTime=1691315703113 2023-08-06 09:55:03,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:03,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:03,132 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=976d2fa78f57b6e8c06e1e70f4d3ffe5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,132 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315703132"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315703132"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315703132"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315703132"}]},"ts":"1691315703132"} 2023-08-06 09:55:03,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-08-06 09:55:03,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 976d2fa78f57b6e8c06e1e70f4d3ffe5, server=jenkins-hbase4.apache.org,40847,1691315596668 in 171 msec 2023-08-06 09:55:03,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-08-06 09:55:03,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=976d2fa78f57b6e8c06e1e70f4d3ffe5, ASSIGN in 328 msec 2023-08-06 09:55:03,136 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-06 09:55:03,136 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315703136"}]},"ts":"1691315703136"} 2023-08-06 09:55:03,137 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-08-06 09:55:03,139 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-08-06 09:55:03,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 370 msec 2023-08-06 09:55:03,276 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:03,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-06 09:55:03,278 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-06 09:55:03,278 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,278 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-06 09:55:03,278 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-06 09:55:03,278 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,278 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,279 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,279 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,279 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,279 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,280 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,280 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,281 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,282 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,282 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:03,282 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,282 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:03,282 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:03,282 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:03,282 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:03,282 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. started... 2023-08-06 09:55:03,283 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 976d2fa78f57b6e8c06e1e70f4d3ffe5 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-06 09:55:03,284 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,285 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,285 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,285 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:03,285 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,285 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:03,292 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed 2023-08-06 09:55:03,303 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/67109c8b8d2444ef91b88f9c9c2e416a 2023-08-06 09:55:03,306 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed 2023-08-06 09:55:03,309 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:55:03,310 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/67109c8b8d2444ef91b88f9c9c2e416a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a 2023-08-06 09:55:03,312 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:55:03,313 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 976d2fa78f57b6e8c06e1e70f4d3ffe5 in 30ms, sequenceid=6, compaction requested=false 2023-08-06 09:55:03,313 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-08-06 09:55:03,314 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 976d2fa78f57b6e8c06e1e70f4d3ffe5: 2023-08-06 09:55:03,314 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:03,314 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:03,314 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:03,314 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,314 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:03,314 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:03,315 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,315 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:03,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:03,316 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,316 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:03,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:03,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:03,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:03,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-06 09:55:03,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d967533[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:03,317 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-06 09:55:03,317 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,318 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,318 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,319 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,319 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:03,319 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,319 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:03,319 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:03,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,319 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:03,319 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:03,319 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:03,319 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:03,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,319 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:03,319 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:03,319 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:03,321 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,321 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:03,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:03,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:03,321 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,321 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:03,325 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:03,325 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:03,325 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:03,325 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,325 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:03,325 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:03,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:03,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:03,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:55:03,327 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-06 09:55:03,327 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-06 09:55:03,327 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:03,327 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:03,327 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-06 09:55:03,327 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:03,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,329 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,330 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,330 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:03,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:03,332 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,332 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,332 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,336 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,336 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:03,336 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:03,337 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:03,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,336 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,337 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,337 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,337 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,337 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,437 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:03,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:03,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:03,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-06 09:55:03,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-06 09:55:03,449 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-06 09:55:03,449 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,449 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-06 09:55:03,449 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-06 09:55:03,449 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,450 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,451 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,451 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,451 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,451 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,451 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,451 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:03,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,452 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,452 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,453 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,453 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:03,453 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:03,453 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:03,453 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:03,453 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. started... 2023-08-06 09:55:03,453 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 976d2fa78f57b6e8c06e1e70f4d3ffe5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:55:03,454 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,454 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,455 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,455 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:03,455 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,455 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:03,466 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/161d0109df9f4a60bbff180b01fd8d11 2023-08-06 09:55:03,478 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa 2023-08-06 09:55:03,482 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/161d0109df9f4a60bbff180b01fd8d11 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11 2023-08-06 09:55:03,485 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:55:03,486 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa 2023-08-06 09:55:03,489 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa, entries=10, sequenceid=19, filesize=5.0 K 2023-08-06 09:55:03,490 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 976d2fa78f57b6e8c06e1e70f4d3ffe5 in 37ms, sequenceid=19, compaction requested=false 2023-08-06 09:55:03,490 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 976d2fa78f57b6e8c06e1e70f4d3ffe5: 2023-08-06 09:55:03,490 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:03,490 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:03,490 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:03,490 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,490 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:03,490 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:03,492 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,492 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:03,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:03,492 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,492 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:03,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:03,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:03,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:03,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-06 09:55:03,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33f03e82[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:03,494 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-06 09:55:03,494 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,496 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,496 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,496 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,496 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:03,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,496 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:03,496 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,496 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:03,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,496 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:03,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,496 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:03,496 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:03,496 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:03,497 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:03,497 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:03,496 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:03,497 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:03,499 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:03,499 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,499 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:03,499 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,499 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:03,499 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,499 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:03,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:03,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:03,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:03,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:03,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:03,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:55:03,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-06 09:55:03,502 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-06 09:55:03,502 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:03,502 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:03,502 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-06 09:55:03,502 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:03,506 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,506 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:03,506 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,506 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,506 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,507 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,506 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,507 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,508 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:03,508 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:03,510 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,510 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,510 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,513 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,513 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,513 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,513 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,513 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,513 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,513 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,513 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-06 09:55:03,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:03,513 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,513 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:03,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,514 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:03,513 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,514 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,514 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,514 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,514 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,514 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,514 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,514 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,514 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,614 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:03,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:03,625 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:03,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-06 09:55:03,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-06 09:55:03,627 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-06 09:55:03,627 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,627 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-06 09:55:03,627 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-06 09:55:03,627 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,627 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,628 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,628 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,628 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,628 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,629 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,629 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:03,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:03,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,630 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,630 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,630 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:03,630 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,630 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,630 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,631 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:03,631 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:03,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,631 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:03,631 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:03,631 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:03,631 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. started... 2023-08-06 09:55:03,631 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,631 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,631 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 976d2fa78f57b6e8c06e1e70f4d3ffe5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:55:03,631 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,631 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:03,631 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:03,632 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,806 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,806 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,806 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:03,807 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,807 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:03,811 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/7b0ef96d983143ea95d26081055536a4 2023-08-06 09:55:03,822 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/035c775e14274988ae52bbd7832bb373 2023-08-06 09:55:03,826 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/7b0ef96d983143ea95d26081055536a4 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4 2023-08-06 09:55:03,829 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:55:03,830 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/035c775e14274988ae52bbd7832bb373 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373 2023-08-06 09:55:03,833 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373, entries=10, sequenceid=32, filesize=5.0 K 2023-08-06 09:55:03,834 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 976d2fa78f57b6e8c06e1e70f4d3ffe5 in 203ms, sequenceid=32, compaction requested=true 2023-08-06 09:55:03,834 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 976d2fa78f57b6e8c06e1e70f4d3ffe5: 2023-08-06 09:55:03,834 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:03,834 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:03,834 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:03,834 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,834 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:03,834 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:03,836 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,836 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:03,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:03,836 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,836 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:03,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:03,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:03,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:03,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-06 09:55:03,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ec05376[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:03,838 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-06 09:55:03,838 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,839 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,839 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,839 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,839 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:03,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,839 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:03,839 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:03,839 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:03,839 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,839 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,839 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:03,839 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:03,840 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:03,840 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:03,840 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:03,840 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:03,841 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,841 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:03,842 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,842 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:03,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:03,842 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:03,842 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:03,843 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:03,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:03,843 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,843 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:03,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:03,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:03,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:55:03,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-06 09:55:03,846 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-06 09:55:03,846 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:03,846 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:03,846 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-06 09:55:03,846 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:03,847 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,848 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,848 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,848 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,848 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:03,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:03,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,848 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,848 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:03,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,848 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,849 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,850 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:03,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:03,851 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:03,852 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,852 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,854 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,854 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,854 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,854 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,854 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,854 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,854 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:03,854 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,854 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,855 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,855 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:03,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-06 09:55:03,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:03,855 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,855 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:03,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,855 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:03,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:03,856 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:03,856 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:03,856 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:03,857 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,857 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,857 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,957 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:03,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:03,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:03,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-06 09:55:03,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-06 09:55:03,969 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-06 09:55:03,970 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,970 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-06 09:55:03,970 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-06 09:55:03,970 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,970 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,972 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,973 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,972 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:03,972 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:03,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:03,973 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:03,973 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:03,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:03,974 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,974 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:03,974 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,974 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,974 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,974 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,975 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:03,975 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:03,975 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. started... 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:03,975 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 976d2fa78f57b6e8c06e1e70f4d3ffe5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:55:03,976 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,977 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,977 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,977 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:03,977 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:03,977 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:03,987 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/a7c360c8131b45bcb2058ab562d5caec 2023-08-06 09:55:04,001 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/d425ba34896b4982b93b3a9a15c2d255 2023-08-06 09:55:04,004 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/a7c360c8131b45bcb2058ab562d5caec as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec 2023-08-06 09:55:04,007 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:55:04,008 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/d425ba34896b4982b93b3a9a15c2d255 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255 2023-08-06 09:55:04,011 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255, entries=10, sequenceid=45, filesize=5.0 K 2023-08-06 09:55:04,012 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 976d2fa78f57b6e8c06e1e70f4d3ffe5 in 37ms, sequenceid=45, compaction requested=true 2023-08-06 09:55:04,012 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 976d2fa78f57b6e8c06e1e70f4d3ffe5: 2023-08-06 09:55:04,013 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:04,013 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:04,013 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:04,013 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,013 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:04,013 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:04,014 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,014 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:04,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:04,014 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,015 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:04,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:04,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:04,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:04,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-06 09:55:04,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71ed4da1[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:04,016 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-06 09:55:04,017 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,019 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,019 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,019 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,019 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:04,019 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:04,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,019 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:04,019 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,019 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:04,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,019 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:04,019 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:04,019 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:04,019 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:04,021 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:04,021 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,021 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:04,022 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:04,022 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,022 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:04,022 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:04,022 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,022 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:04,027 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:04,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:04,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:04,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:04,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:55:04,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-06 09:55:04,031 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-06 09:55:04,031 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:04,031 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:04,031 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-06 09:55:04,031 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:04,034 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,034 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:04,034 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,034 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,035 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:04,034 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,035 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,035 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,036 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,036 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:04,038 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,038 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,038 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,044 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,044 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,044 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,044 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,044 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,044 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,044 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,044 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-06 09:55:04,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:04,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,044 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,045 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:04,044 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,045 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:04,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,045 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,046 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,145 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:04,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:04,157 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:04,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-06 09:55:04,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-06 09:55:04,158 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-06 09:55:04,158 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:04,158 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-06 09:55:04,158 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-06 09:55:04,159 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,159 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,160 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,160 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,160 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,160 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,160 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,160 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:04,161 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:04,162 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:04,162 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:04,162 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:04,162 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:04,162 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:04,162 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:04,162 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. started... 2023-08-06 09:55:04,162 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 976d2fa78f57b6e8c06e1e70f4d3ffe5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:55:04,164 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,165 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,165 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,165 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:04,165 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,165 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:04,172 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/65069ae6f9494eca84f674c8597f844d 2023-08-06 09:55:04,186 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/92f3d68e559c4010b8fb8388f1ce3df5 2023-08-06 09:55:04,190 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/65069ae6f9494eca84f674c8597f844d as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d 2023-08-06 09:55:04,193 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:55:04,194 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/92f3d68e559c4010b8fb8388f1ce3df5 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5 2023-08-06 09:55:04,198 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5, entries=10, sequenceid=58, filesize=5.0 K 2023-08-06 09:55:04,199 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 976d2fa78f57b6e8c06e1e70f4d3ffe5 in 37ms, sequenceid=58, compaction requested=true 2023-08-06 09:55:04,199 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 976d2fa78f57b6e8c06e1e70f4d3ffe5: 2023-08-06 09:55:04,199 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:04,199 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:04,199 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:04,199 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,199 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:04,199 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:04,200 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,200 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:04,201 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,201 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:04,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:04,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:04,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:04,203 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-06 09:55:04,203 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f7c9142[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:04,203 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-06 09:55:04,203 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,204 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,204 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,204 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,204 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,204 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:04,204 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:04,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,204 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:04,204 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,204 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:04,204 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:04,204 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:04,204 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:04,204 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:04,204 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:04,207 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:04,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:55:04,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-06 09:55:04,211 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-06 09:55:04,211 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:04,211 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:04,211 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-06 09:55:04,211 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:04,212 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,212 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,212 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,212 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:04,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:04,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,212 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,212 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:04,212 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,213 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,214 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,214 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,216 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,216 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,216 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,218 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,218 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,219 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-06 09:55:04,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:04,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,219 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,220 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,220 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,220 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:04,220 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:04,220 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,220 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,220 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,220 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:04,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:04,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegion because node does not exist (not an error) 2023-08-06 09:55:04,320 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:04,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:04,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-06 09:55:04,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-06 09:55:04,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-06 09:55:04,335 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-06 09:55:04,335 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:04,335 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-06 09:55:04,335 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-06 09:55:04,336 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,336 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,337 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,338 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,338 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,338 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,338 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,338 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-06 09:55:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:04,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:04,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-06 09:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:04,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-06 09:55:04,340 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:04,340 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:04,340 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:04,342 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:04,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:04,342 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:04,343 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:04,343 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-06 09:55:04,343 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:04,343 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:04,343 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. started... 2023-08-06 09:55:04,343 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:04,343 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39585,1691315596578' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:04,343 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-06 09:55:04,343 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 976d2fa78f57b6e8c06e1e70f4d3ffe5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-06 09:55:04,345 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,345 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,345 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:04,346 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-06 09:55:04,346 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-06 09:55:04,347 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-06 09:55:04,347 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-06 09:55:04,347 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-06 09:55:04,347 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:04,347 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44015,1691315596630' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:04,348 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,348 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,348 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:04,362 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/848fe8f1e50944379ba9f248ef951f34 2023-08-06 09:55:04,377 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/791605881e3a47efbfaa5c4993065e14 2023-08-06 09:55:04,380 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/848fe8f1e50944379ba9f248ef951f34 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34 2023-08-06 09:55:04,384 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:55:04,384 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/791605881e3a47efbfaa5c4993065e14 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14 2023-08-06 09:55:04,388 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14, entries=10, sequenceid=71, filesize=5.0 K 2023-08-06 09:55:04,388 INFO [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 976d2fa78f57b6e8c06e1e70f4d3ffe5 in 45ms, sequenceid=71, compaction requested=true 2023-08-06 09:55:04,388 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 976d2fa78f57b6e8c06e1e70f4d3ffe5: 2023-08-06 09:55:04,388 DEBUG [rs(jenkins-hbase4.apache.org,40847,1691315596668)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:04,389 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-06 09:55:04,389 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-06 09:55:04,389 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,389 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-06 09:55:04,389 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-06 09:55:04,390 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,390 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:04,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:04,391 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:04,391 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-06 09:55:04,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:04,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:04,392 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40847,1691315596668' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-06 09:55:04,392 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d4aa001[Count = 0] remaining members to acquire global barrier 2023-08-06 09:55:04,392 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-06 09:55:04,392 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,393 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,393 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,393 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,394 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,394 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-06 09:55:04,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,394 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:04,394 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:04,394 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:04,394 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:04,394 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39585,1691315596578' in zk 2023-08-06 09:55:04,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,394 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44015,1691315596630' in zk 2023-08-06 09:55:04,394 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-06 09:55:04,394 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-06 09:55:04,394 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40847,1691315596668' in zk 2023-08-06 09:55:04,398 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:04,398 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:04,399 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,398 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,399 DEBUG [member: 'jenkins-hbase4.apache.org,44015,1691315596630' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:04,399 DEBUG [member: 'jenkins-hbase4.apache.org,39585,1691315596578' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:04,399 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,399 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-06 09:55:04,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:04,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:04,399 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,399 DEBUG [member: 'jenkins-hbase4.apache.org,40847,1691315596668' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-06 09:55:04,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:04,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:04,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:04,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40847,1691315596668': 2023-08-06 09:55:04,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40847,1691315596668' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-06 09:55:04,402 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-06 09:55:04,403 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-06 09:55:04,403 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-06 09:55:04,403 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-06 09:55:04,403 INFO [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-06 09:55:04,404 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,404 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,404 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,404 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,404 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-06 09:55:04,404 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-06 09:55:04,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,405 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-06 09:55:04,405 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,406 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-06 09:55:04,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,406 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-06 09:55:04,408 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-06 09:55:04,408 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,409 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,412 DEBUG [(jenkins-hbase4.apache.org,33207,1691315595337)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-06 09:55:04,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-06 09:55:04,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-06 09:55:04,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-06 09:55:04,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-06 09:55:04,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-06 09:55:04,412 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-06 09:55:04,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:04,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:04,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-06 09:55:04,413 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-06 09:55:04,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-06 09:55:04,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-06 09:55:04,512 DEBUG [Listener at localhost/36789] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-06 09:55:04,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-06 09:55:04,520 DEBUG [Listener at localhost/36789] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,521 DEBUG [Listener at localhost/36789] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=976d2fa78f57b6e8c06e1e70f4d3ffe5, daughterA=fbbf8cfc9a6a2904f6c4246e5b93b4a7, daughterB=72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:04,522 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=976d2fa78f57b6e8c06e1e70f4d3ffe5, daughterA=fbbf8cfc9a6a2904f6c4246e5b93b4a7, daughterB=72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:04,522 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=976d2fa78f57b6e8c06e1e70f4d3ffe5, daughterA=fbbf8cfc9a6a2904f6c4246e5b93b4a7, daughterB=72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:04,522 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=976d2fa78f57b6e8c06e1e70f4d3ffe5, daughterA=fbbf8cfc9a6a2904f6c4246e5b93b4a7, daughterB=72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:04,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=976d2fa78f57b6e8c06e1e70f4d3ffe5, UNASSIGN}] 2023-08-06 09:55:04,524 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=976d2fa78f57b6e8c06e1e70f4d3ffe5, UNASSIGN 2023-08-06 09:55:04,525 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=976d2fa78f57b6e8c06e1e70f4d3ffe5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:04,525 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315704525"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315704525"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315704525"}]},"ts":"1691315704525"} 2023-08-06 09:55:04,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 976d2fa78f57b6e8c06e1e70f4d3ffe5, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:55:04,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 976d2fa78f57b6e8c06e1e70f4d3ffe5, disabling compactions & flushes 2023-08-06 09:55:04,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:04,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:04,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. after waiting 0 ms 2023-08-06 09:55:04,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:04,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 976d2fa78f57b6e8c06e1e70f4d3ffe5 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-06 09:55:04,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/b9353ca3bf0b464c934aa4f8c74486a1 2023-08-06 09:55:04,698 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/8a619b2340864aee807345a29d44d70f 2023-08-06 09:55:04,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf1/b9353ca3bf0b464c934aa4f8c74486a1 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1 2023-08-06 09:55:04,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:55:04,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/.tmp/cf2/8a619b2340864aee807345a29d44d70f as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f 2023-08-06 09:55:04,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f, entries=8, sequenceid=82, filesize=5.0 K 2023-08-06 09:55:04,710 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 976d2fa78f57b6e8c06e1e70f4d3ffe5 in 32ms, sequenceid=82, compaction requested=true 2023-08-06 09:55:04,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-06 09:55:04,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:55:04,716 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. 2023-08-06 09:55:04,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 976d2fa78f57b6e8c06e1e70f4d3ffe5: 2023-08-06 09:55:04,717 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,718 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=976d2fa78f57b6e8c06e1e70f4d3ffe5, regionState=CLOSED 2023-08-06 09:55:04,718 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315704718"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315704718"}]},"ts":"1691315704718"} 2023-08-06 09:55:04,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-08-06 09:55:04,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 976d2fa78f57b6e8c06e1e70f4d3ffe5, server=jenkins-hbase4.apache.org,40847,1691315596668 in 193 msec 2023-08-06 09:55:04,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-08-06 09:55:04,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=976d2fa78f57b6e8c06e1e70f4d3ffe5, UNASSIGN in 196 msec 2023-08-06 09:55:04,732 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=976d2fa78f57b6e8c06e1e70f4d3ffe5, threads=14 2023-08-06 09:55:04,732 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,732 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,735 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,735 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,735 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,735 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,735 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,735 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,735 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,736 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,736 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,736 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,736 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,737 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,923 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,936 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,937 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,937 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,951 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,955 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,960 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,970 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,975 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,975 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,979 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,986 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,987 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,989 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1 for region: 976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:04,989 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 976d2fa78f57b6e8c06e1e70f4d3ffe5 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-06 09:55:05,021 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:55:05,023 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-06 09:55:05,024 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1691315705024"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1691315705024"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1691315705024"}]},"ts":"1691315705024"} 2023-08-06 09:55:05,024 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315705024"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315705024"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315705024"}]},"ts":"1691315705024"} 2023-08-06 09:55:05,024 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315705024"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315705024"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315705024"}]},"ts":"1691315705024"} 2023-08-06 09:55:05,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=fbbf8cfc9a6a2904f6c4246e5b93b4a7, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=72d256961d260ce7f79aafabda19f32f, ASSIGN}] 2023-08-06 09:55:05,028 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=72d256961d260ce7f79aafabda19f32f, ASSIGN 2023-08-06 09:55:05,028 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=72d256961d260ce7f79aafabda19f32f, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40847,1691315596668; forceNewPlan=false, retain=false 2023-08-06 09:55:05,029 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=fbbf8cfc9a6a2904f6c4246e5b93b4a7, ASSIGN 2023-08-06 09:55:05,029 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=fbbf8cfc9a6a2904f6c4246e5b93b4a7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40847,1691315596668; forceNewPlan=false, retain=false 2023-08-06 09:55:05,044 ERROR [BP-1515658381-172.31.14.131-1691315591457 heartbeating to localhost/127.0.0.1:43345] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1515658381-172.31.14.131-1691315591457 (Datanode Uuid 66ee7809-9841-425b-bf64-0316ea0c97c6) service to localhost/127.0.0.1:43345 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-06 09:55:05,179 INFO [jenkins-hbase4:33207] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-06 09:55:05,181 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=72d256961d260ce7f79aafabda19f32f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:05,181 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315705180"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315705180"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315705180"}]},"ts":"1691315705180"} 2023-08-06 09:55:05,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 72d256961d260ce7f79aafabda19f32f, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:55:05,184 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=fbbf8cfc9a6a2904f6c4246e5b93b4a7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:05,184 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315705184"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315705184"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315705184"}]},"ts":"1691315705184"} 2023-08-06 09:55:05,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure fbbf8cfc9a6a2904f6c4246e5b93b4a7, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:55:05,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. 2023-08-06 09:55:05,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 72d256961d260ce7f79aafabda19f32f, NAME => 'testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.', STARTKEY => '41', ENDKEY => ''} 2023-08-06 09:55:05,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:55:05,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:05,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:55:05,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:05,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:05,338 INFO [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:05,339 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1 2023-08-06 09:55:05,339 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1 2023-08-06 09:55:05,340 INFO [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 72d256961d260ce7f79aafabda19f32f columnFamilyName cf1 2023-08-06 09:55:05,353 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11-top 2023-08-06 09:55:05,357 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed-top 2023-08-06 09:55:05,360 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d-top 2023-08-06 09:55:05,364 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4-top 2023-08-06 09:55:05,368 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34-top 2023-08-06 09:55:05,375 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec-top 2023-08-06 09:55:05,378 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1-top 2023-08-06 09:55:05,378 INFO [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(310): Store=72d256961d260ce7f79aafabda19f32f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:05,379 INFO [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:05,379 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2 2023-08-06 09:55:05,379 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2 2023-08-06 09:55:05,380 INFO [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 72d256961d260ce7f79aafabda19f32f columnFamilyName cf2 2023-08-06 09:55:05,392 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373-top 2023-08-06 09:55:05,396 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa-top 2023-08-06 09:55:05,400 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a-top 2023-08-06 09:55:05,404 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14-top 2023-08-06 09:55:05,407 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f-top 2023-08-06 09:55:05,410 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5-top 2023-08-06 09:55:05,414 DEBUG [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255-top 2023-08-06 09:55:05,414 INFO [StoreOpener-72d256961d260ce7f79aafabda19f32f-1] regionserver.HStore(310): Store=72d256961d260ce7f79aafabda19f32f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:05,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:05,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:05,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:55:05,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:05,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 72d256961d260ce7f79aafabda19f32f; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61752231, jitterRate=-0.07982005178928375}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:55:05,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 72d256961d260ce7f79aafabda19f32f: 2023-08-06 09:55:05,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f., pid=149, masterSystemTime=1691315705333 2023-08-06 09:55:05,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:55:05,421 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:55:05,421 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:55:05,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:55:05,421 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. 2023-08-06 09:55:05,421 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1912): 72d256961d260ce7f79aafabda19f32f/cf2 is initiating minor compaction (all files) 2023-08-06 09:55:05,421 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. 2023-08-06 09:55:05,421 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 72d256961d260ce7f79aafabda19f32f/cf2 in testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. 2023-08-06 09:55:05,421 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1912): 72d256961d260ce7f79aafabda19f32f/cf1 is initiating minor compaction (all files) 2023-08-06 09:55:05,421 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 72d256961d260ce7f79aafabda19f32f/cf1 in testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. 2023-08-06 09:55:05,421 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f-top] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/.tmp, totalSize=34.9 K 2023-08-06 09:55:05,422 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1-top] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/.tmp, totalSize=34.9 K 2023-08-06 09:55:05,422 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691315703272 2023-08-06 09:55:05,422 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1691315703272 2023-08-06 09:55:05,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. 2023-08-06 09:55:05,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. 2023-08-06 09:55:05,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. 2023-08-06 09:55:05,423 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691315703438 2023-08-06 09:55:05,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fbbf8cfc9a6a2904f6c4246e5b93b4a7, NAME => 'testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.', STARTKEY => '', ENDKEY => '41'} 2023-08-06 09:55:05,423 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=72d256961d260ce7f79aafabda19f32f, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:05,423 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315705423"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315705423"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315705423"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315705423"}]},"ts":"1691315705423"} 2023-08-06 09:55:05,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-06 09:55:05,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:05,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-06 09:55:05,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:05,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:05,424 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1691315703438 2023-08-06 09:55:05,424 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691315703615 2023-08-06 09:55:05,424 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1691315703615 2023-08-06 09:55:05,425 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691315703958 2023-08-06 09:55:05,425 INFO [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:05,426 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1691315703958 2023-08-06 09:55:05,426 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691315704147 2023-08-06 09:55:05,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-08-06 09:55:05,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 72d256961d260ce7f79aafabda19f32f, server=jenkins-hbase4.apache.org,40847,1691315596668 in 242 msec 2023-08-06 09:55:05,426 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1691315704147 2023-08-06 09:55:05,426 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691315704322 2023-08-06 09:55:05,426 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1 2023-08-06 09:55:05,427 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1 2023-08-06 09:55:05,427 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1691315704322 2023-08-06 09:55:05,427 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691315704514 2023-08-06 09:55:05,427 INFO [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fbbf8cfc9a6a2904f6c4246e5b93b4a7 columnFamilyName cf1 2023-08-06 09:55:05,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=72d256961d260ce7f79aafabda19f32f, ASSIGN in 399 msec 2023-08-06 09:55:05,428 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1691315704514 2023-08-06 09:55:05,445 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] throttle.PressureAwareThroughputController(145): 72d256961d260ce7f79aafabda19f32f#cf1#compaction#143 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:55:05,445 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] throttle.PressureAwareThroughputController(145): 72d256961d260ce7f79aafabda19f32f#cf2#compaction#142 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:55:05,448 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11-bottom 2023-08-06 09:55:05,451 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed-bottom 2023-08-06 09:55:05,454 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d-bottom 2023-08-06 09:55:05,457 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4-bottom 2023-08-06 09:55:05,460 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34-bottom 2023-08-06 09:55:05,476 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec-bottom 2023-08-06 09:55:05,479 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/.tmp/cf2/576724df9d194407880c84bcb18a4d2a as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/576724df9d194407880c84bcb18a4d2a 2023-08-06 09:55:05,480 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1-bottom 2023-08-06 09:55:05,480 INFO [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(310): Store=fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:05,480 INFO [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:05,481 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2 2023-08-06 09:55:05,481 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2 2023-08-06 09:55:05,481 INFO [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fbbf8cfc9a6a2904f6c4246e5b93b4a7 columnFamilyName cf2 2023-08-06 09:55:05,482 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/.tmp/cf1/07c40b6bfcf442beb1785ae6c0f6da74 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/07c40b6bfcf442beb1785ae6c0f6da74 2023-08-06 09:55:05,484 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 72d256961d260ce7f79aafabda19f32f/cf2 of 72d256961d260ce7f79aafabda19f32f into 576724df9d194407880c84bcb18a4d2a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:55:05,484 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 72d256961d260ce7f79aafabda19f32f: 2023-08-06 09:55:05,484 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f., storeName=72d256961d260ce7f79aafabda19f32f/cf2, priority=9, startTime=1691315705421; duration=0sec 2023-08-06 09:55:05,484 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:55:05,487 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 72d256961d260ce7f79aafabda19f32f/cf1 of 72d256961d260ce7f79aafabda19f32f into 07c40b6bfcf442beb1785ae6c0f6da74(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:55:05,487 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 72d256961d260ce7f79aafabda19f32f: 2023-08-06 09:55:05,487 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f., storeName=72d256961d260ce7f79aafabda19f32f/cf1, priority=9, startTime=1691315705420; duration=0sec 2023-08-06 09:55:05,487 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:55:05,493 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373-bottom 2023-08-06 09:55:05,496 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa-bottom 2023-08-06 09:55:05,498 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a-bottom 2023-08-06 09:55:05,502 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14-bottom 2023-08-06 09:55:05,505 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f-bottom 2023-08-06 09:55:05,508 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5-bottom 2023-08-06 09:55:05,511 DEBUG [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(539): loaded hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255-bottom 2023-08-06 09:55:05,512 INFO [StoreOpener-fbbf8cfc9a6a2904f6c4246e5b93b4a7-1] regionserver.HStore(310): Store=fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-06 09:55:05,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:05,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:05,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-06 09:55:05,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:05,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fbbf8cfc9a6a2904f6c4246e5b93b4a7; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64071185, jitterRate=-0.045264944434165955}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-06 09:55:05,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fbbf8cfc9a6a2904f6c4246e5b93b4a7: 2023-08-06 09:55:05,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7., pid=150, masterSystemTime=1691315705333 2023-08-06 09:55:05,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:55:05,518 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:55:05,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:55:05,518 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-06 09:55:05,518 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. 2023-08-06 09:55:05,518 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1912): fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1 is initiating minor compaction (all files) 2023-08-06 09:55:05,518 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. 2023-08-06 09:55:05,518 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1 in testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. 2023-08-06 09:55:05,518 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1912): fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2 is initiating minor compaction (all files) 2023-08-06 09:55:05,518 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2259): Starting compaction of fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2 in testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. 2023-08-06 09:55:05,518 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1-bottom] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/.tmp, totalSize=34.9 K 2023-08-06 09:55:05,518 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f-bottom] into tmpdir=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/.tmp, totalSize=34.9 K 2023-08-06 09:55:05,519 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315703272 2023-08-06 09:55:05,519 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1691315703272 2023-08-06 09:55:05,520 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315703438 2023-08-06 09:55:05,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. 2023-08-06 09:55:05,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. 2023-08-06 09:55:05,520 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1691315703438 2023-08-06 09:55:05,520 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315703615 2023-08-06 09:55:05,520 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=fbbf8cfc9a6a2904f6c4246e5b93b4a7, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:05,520 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315703958 2023-08-06 09:55:05,520 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1691315703615 2023-08-06 09:55:05,520 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315705520"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691315705520"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691315705520"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691315705520"}]},"ts":"1691315705520"} 2023-08-06 09:55:05,521 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315704147 2023-08-06 09:55:05,521 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1691315703958 2023-08-06 09:55:05,521 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting 848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315704322 2023-08-06 09:55:05,521 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1691315704147 2023-08-06 09:55:05,522 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] compactions.Compactor(207): Compacting b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315704514 2023-08-06 09:55:05,522 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1691315704322 2023-08-06 09:55:05,522 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] compactions.Compactor(207): Compacting 8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1691315704514 2023-08-06 09:55:05,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-08-06 09:55:05,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure fbbf8cfc9a6a2904f6c4246e5b93b4a7, server=jenkins-hbase4.apache.org,40847,1691315596668 in 337 msec 2023-08-06 09:55:05,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-08-06 09:55:05,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=fbbf8cfc9a6a2904f6c4246e5b93b4a7, ASSIGN in 496 msec 2023-08-06 09:55:05,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=976d2fa78f57b6e8c06e1e70f4d3ffe5, daughterA=fbbf8cfc9a6a2904f6c4246e5b93b4a7, daughterB=72d256961d260ce7f79aafabda19f32f in 1.0050 sec 2023-08-06 09:55:05,532 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] throttle.PressureAwareThroughputController(145): fbbf8cfc9a6a2904f6c4246e5b93b4a7#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:55:05,532 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] throttle.PressureAwareThroughputController(145): fbbf8cfc9a6a2904f6c4246e5b93b4a7#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-06 09:55:05,547 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/.tmp/cf1/8bd5c99004754e62b5a9798668f57a17 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/8bd5c99004754e62b5a9798668f57a17 2023-08-06 09:55:05,548 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/.tmp/cf2/4846de9199634201b95ae902975a0072 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/4846de9199634201b95ae902975a0072 2023-08-06 09:55:05,551 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1 of fbbf8cfc9a6a2904f6c4246e5b93b4a7 into 8bd5c99004754e62b5a9798668f57a17(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:55:05,551 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fbbf8cfc9a6a2904f6c4246e5b93b4a7: 2023-08-06 09:55:05,551 INFO [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7., storeName=fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1, priority=9, startTime=1691315705517; duration=0sec 2023-08-06 09:55:05,551 DEBUG [RS:2;jenkins-hbase4:40847-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:55:05,552 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2 of fbbf8cfc9a6a2904f6c4246e5b93b4a7 into 4846de9199634201b95ae902975a0072(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-06 09:55:05,552 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for fbbf8cfc9a6a2904f6c4246e5b93b4a7: 2023-08-06 09:55:05,552 INFO [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7., storeName=fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2, priority=9, startTime=1691315705518; duration=0sec 2023-08-06 09:55:05,552 DEBUG [RS:2;jenkins-hbase4:40847-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-06 09:55:05,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40847] ipc.CallRunner(144): callId: 1122 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:49972 deadline: 1691315765772, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5. is not online on jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:05,883 DEBUG [Listener at localhost/36789] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-06 09:55:05,889 INFO [Listener at localhost/36789] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-06 09:55:05,891 DEBUG [Listener at localhost/36789] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-06 09:55:05,898 WARN [Listener at localhost/36789] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-06 09:55:05,900 INFO [Listener at localhost/36789] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-08-06 09:55:05,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-08-06 09:55:05,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-08-06 09:55:05,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-08-06 09:55:05,904 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315705903"}]},"ts":"1691315705903"} 2023-08-06 09:55:05,904 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-08-06 09:55:05,907 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-08-06 09:55:05,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=fbbf8cfc9a6a2904f6c4246e5b93b4a7, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=72d256961d260ce7f79aafabda19f32f, UNASSIGN}] 2023-08-06 09:55:05,908 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=72d256961d260ce7f79aafabda19f32f, UNASSIGN 2023-08-06 09:55:05,908 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=72d256961d260ce7f79aafabda19f32f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:05,909 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315705908"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315705908"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315705908"}]},"ts":"1691315705908"} 2023-08-06 09:55:05,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 72d256961d260ce7f79aafabda19f32f, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:55:05,911 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=fbbf8cfc9a6a2904f6c4246e5b93b4a7, UNASSIGN 2023-08-06 09:55:05,912 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=fbbf8cfc9a6a2904f6c4246e5b93b4a7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:05,912 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315705912"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691315705912"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691315705912"}]},"ts":"1691315705912"} 2023-08-06 09:55:05,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure fbbf8cfc9a6a2904f6c4246e5b93b4a7, server=jenkins-hbase4.apache.org,40847,1691315596668}] 2023-08-06 09:55:06,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-08-06 09:55:06,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:06,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 72d256961d260ce7f79aafabda19f32f, disabling compactions & flushes 2023-08-06 09:55:06,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. 2023-08-06 09:55:06,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. 2023-08-06 09:55:06,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. after waiting 0 ms 2023-08-06 09:55:06,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. 2023-08-06 09:55:06,062 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1-top] to archive 2023-08-06 09:55:06,063 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:55:06,065 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,066 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,067 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,068 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,069 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,070 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,071 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,072 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14-top, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f-top] to archive 2023-08-06 09:55:06,072 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:55:06,074 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,075 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,076 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,077 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,078 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,079 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,079 DEBUG [StoreCloser-testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,082 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-06 09:55:06,083 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:55:06,083 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f. 2023-08-06 09:55:06,083 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 72d256961d260ce7f79aafabda19f32f: 2023-08-06 09:55:06,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:06,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:06,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fbbf8cfc9a6a2904f6c4246e5b93b4a7, disabling compactions & flushes 2023-08-06 09:55:06,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. 2023-08-06 09:55:06,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. 2023-08-06 09:55:06,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. after waiting 0 ms 2023-08-06 09:55:06,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. 2023-08-06 09:55:06,085 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=72d256961d260ce7f79aafabda19f32f, regionState=CLOSED 2023-08-06 09:55:06,085 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315706085"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315706085"}]},"ts":"1691315706085"} 2023-08-06 09:55:06,085 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1-bottom] to archive 2023-08-06 09:55:06,086 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:55:06,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-08-06 09:55:06,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 72d256961d260ce7f79aafabda19f32f, server=jenkins-hbase4.apache.org,40847,1691315596668 in 177 msec 2023-08-06 09:55:06,088 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=72d256961d260ce7f79aafabda19f32f, UNASSIGN in 181 msec 2023-08-06 09:55:06,089 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/161d0109df9f4a60bbff180b01fd8d11.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,090 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/7b0ef96d983143ea95d26081055536a4.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,091 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/a7c360c8131b45bcb2058ab562d5caec.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,092 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/65069ae6f9494eca84f674c8597f844d.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,093 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/848fe8f1e50944379ba9f248ef951f34.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,093 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/b9353ca3bf0b464c934aa4f8c74486a1.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,095 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14-bottom, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5->hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f-bottom] to archive 2023-08-06 09:55:06,096 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-06 09:55:06,097 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/67109c8b8d2444ef91b88f9c9c2e416a.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,098 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,099 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/035c775e14274988ae52bbd7832bb373.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,100 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/d425ba34896b4982b93b3a9a15c2d255.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,101 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/92f3d68e559c4010b8fb8388f1ce3df5.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,102 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/791605881e3a47efbfaa5c4993065e14.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,103 DEBUG [StoreCloser-testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/8a619b2340864aee807345a29d44d70f.976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-06 09:55:06,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:55:06,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7. 2023-08-06 09:55:06,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fbbf8cfc9a6a2904f6c4246e5b93b4a7: 2023-08-06 09:55:06,108 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:06,109 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=fbbf8cfc9a6a2904f6c4246e5b93b4a7, regionState=CLOSED 2023-08-06 09:55:06,109 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1691315706109"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691315706109"}]},"ts":"1691315706109"} 2023-08-06 09:55:06,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-08-06 09:55:06,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure fbbf8cfc9a6a2904f6c4246e5b93b4a7, server=jenkins-hbase4.apache.org,40847,1691315596668 in 197 msec 2023-08-06 09:55:06,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-08-06 09:55:06,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=fbbf8cfc9a6a2904f6c4246e5b93b4a7, UNASSIGN in 204 msec 2023-08-06 09:55:06,113 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691315706113"}]},"ts":"1691315706113"} 2023-08-06 09:55:06,113 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-08-06 09:55:06,115 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-08-06 09:55:06,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 215 msec 2023-08-06 09:55:06,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-08-06 09:55:06,206 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-08-06 09:55:06,206 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-08-06 09:55:06,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-08-06 09:55:06,209 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-06 09:55:06,209 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-06 09:55:06,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-08-06 09:55:06,213 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,213 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:06,213 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:06,215 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/recovered.edits] 2023-08-06 09:55:06,215 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/recovered.edits] 2023-08-06 09:55:06,215 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2, FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/recovered.edits] 2023-08-06 09:55:06,218 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/07c40b6bfcf442beb1785ae6c0f6da74 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf1/07c40b6bfcf442beb1785ae6c0f6da74 2023-08-06 09:55:06,218 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/8bd5c99004754e62b5a9798668f57a17 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf1/8bd5c99004754e62b5a9798668f57a17 2023-08-06 09:55:06,219 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/161d0109df9f4a60bbff180b01fd8d11 2023-08-06 09:55:06,220 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/576724df9d194407880c84bcb18a4d2a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/cf2/576724df9d194407880c84bcb18a4d2a 2023-08-06 09:55:06,220 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/4ae937fd72ce48c0ba1a4b2f1dbaefed 2023-08-06 09:55:06,220 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/4846de9199634201b95ae902975a0072 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/cf2/4846de9199634201b95ae902975a0072 2023-08-06 09:55:06,222 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/65069ae6f9494eca84f674c8597f844d 2023-08-06 09:55:06,223 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/recovered.edits/90.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f/recovered.edits/90.seqid 2023-08-06 09:55:06,223 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/7b0ef96d983143ea95d26081055536a4 2023-08-06 09:55:06,223 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/recovered.edits/90.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7/recovered.edits/90.seqid 2023-08-06 09:55:06,224 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/72d256961d260ce7f79aafabda19f32f 2023-08-06 09:55:06,224 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/fbbf8cfc9a6a2904f6c4246e5b93b4a7 2023-08-06 09:55:06,224 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/848fe8f1e50944379ba9f248ef951f34 2023-08-06 09:55:06,226 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/a7c360c8131b45bcb2058ab562d5caec 2023-08-06 09:55:06,227 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf1/b9353ca3bf0b464c934aa4f8c74486a1 2023-08-06 09:55:06,229 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/035c775e14274988ae52bbd7832bb373 2023-08-06 09:55:06,230 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/624b8fe13ff84ffe8eb0b6d5e884c2fa 2023-08-06 09:55:06,231 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/67109c8b8d2444ef91b88f9c9c2e416a 2023-08-06 09:55:06,232 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/791605881e3a47efbfaa5c4993065e14 2023-08-06 09:55:06,233 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/8a619b2340864aee807345a29d44d70f 2023-08-06 09:55:06,234 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/92f3d68e559c4010b8fb8388f1ce3df5 2023-08-06 09:55:06,235 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255 to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/cf2/d425ba34896b4982b93b3a9a15c2d255 2023-08-06 09:55:06,237 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/recovered.edits/85.seqid to hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/archive/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5/recovered.edits/85.seqid 2023-08-06 09:55:06,237 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/.tmp/data/default/testSplitTableRegion/976d2fa78f57b6e8c06e1e70f4d3ffe5 2023-08-06 09:55:06,237 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-08-06 09:55:06,239 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-06 09:55:06,240 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-08-06 09:55:06,242 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-08-06 09:55:06,243 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-06 09:55:06,243 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-08-06 09:55:06,243 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315706243"}]},"ts":"9223372036854775807"} 2023-08-06 09:55:06,243 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315706243"}]},"ts":"9223372036854775807"} 2023-08-06 09:55:06,243 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1691315706243"}]},"ts":"9223372036854775807"} 2023-08-06 09:55:06,244 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-06 09:55:06,244 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 976d2fa78f57b6e8c06e1e70f4d3ffe5, NAME => 'testSplitTableRegion,,1691315702769.976d2fa78f57b6e8c06e1e70f4d3ffe5.', STARTKEY => '', ENDKEY => ''}, {ENCODED => fbbf8cfc9a6a2904f6c4246e5b93b4a7, NAME => 'testSplitTableRegion,,1691315704520.fbbf8cfc9a6a2904f6c4246e5b93b4a7.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 72d256961d260ce7f79aafabda19f32f, NAME => 'testSplitTableRegion,41,1691315704520.72d256961d260ce7f79aafabda19f32f.', STARTKEY => '41', ENDKEY => ''}] 2023-08-06 09:55:06,245 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-08-06 09:55:06,245 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1691315706245"}]},"ts":"9223372036854775807"} 2023-08-06 09:55:06,246 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-08-06 09:55:06,247 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-06 09:55:06,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 41 msec 2023-08-06 09:55:06,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33207] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-08-06 09:55:06,311 INFO [Listener at localhost/36789] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-08-06 09:55:06,325 INFO [Listener at localhost/36789] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=381 (was 379) - Thread LEAK? -, OpenFileDescriptor=837 (was 835) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=151 (was 151), ProcessCount=171 (was 171), AvailableMemoryMB=2209 (was 2275) 2023-08-06 09:55:06,325 INFO [Listener at localhost/36789] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-06 09:55:06,325 INFO [Listener at localhost/36789] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-06 09:55:06,326 DEBUG [Listener at localhost/36789] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x69c0a999 to 127.0.0.1:52496 2023-08-06 09:55:06,326 DEBUG [Listener at localhost/36789] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-06 09:55:06,327 DEBUG [Listener at localhost/36789] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-06 09:55:06,327 DEBUG [Listener at localhost/36789] util.JVMClusterUtil(257): Found active master hash=741181557, stopped=false 2023-08-06 09:55:06,327 INFO [Listener at localhost/36789] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33207,1691315595337 2023-08-06 09:55:06,328 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-06 09:55:06,328 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-06 09:55:06,328 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-06 09:55:06,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-06 09:55:06,329 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-06 09:55:06,329 INFO [Listener at localhost/36789] procedure2.ProcedureExecutor(629): Stopping 2023-08-06 09:55:06,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-06 09:55:06,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-06 09:55:06,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-06 09:55:06,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-06 09:55:06,330 DEBUG [Listener at localhost/36789] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x59654f68 to 127.0.0.1:52496 2023-08-06 09:55:06,330 DEBUG [Listener at localhost/36789] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-06 09:55:06,330 INFO [Listener at localhost/36789] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39585,1691315596578' ***** 2023-08-06 09:55:06,330 INFO [Listener at localhost/36789] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-06 09:55:06,330 INFO [Listener at localhost/36789] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44015,1691315596630' ***** 2023-08-06 09:55:06,330 INFO [Listener at localhost/36789] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-06 09:55:06,330 INFO [Listener at localhost/36789] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40847,1691315596668' ***** 2023-08-06 09:55:06,330 INFO [Listener at localhost/36789] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-06 09:55:06,330 INFO [RS:0;jenkins-hbase4:39585] regionserver.HeapMemoryManager(220): Stopping 2023-08-06 09:55:06,330 INFO [RS:2;jenkins-hbase4:40847] regionserver.HeapMemoryManager(220): Stopping 2023-08-06 09:55:06,331 INFO [RS:2;jenkins-hbase4:40847] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-06 09:55:06,331 INFO [RS:2;jenkins-hbase4:40847] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-06 09:55:06,331 INFO [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(3305): Received CLOSE for ebbcd508c2d5b7b6263f7d0763491964 2023-08-06 09:55:06,331 INFO [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:06,331 DEBUG [RS:2;jenkins-hbase4:40847] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x70a3177b to 127.0.0.1:52496 2023-08-06 09:55:06,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ebbcd508c2d5b7b6263f7d0763491964, disabling compactions & flushes 2023-08-06 09:55:06,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:55:06,330 INFO [RS:1;jenkins-hbase4:44015] regionserver.HeapMemoryManager(220): Stopping 2023-08-06 09:55:06,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:55:06,331 INFO [RS:1;jenkins-hbase4:44015] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-06 09:55:06,332 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-06 09:55:06,332 INFO [RS:1;jenkins-hbase4:44015] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-06 09:55:06,331 DEBUG [RS:2;jenkins-hbase4:40847] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-06 09:55:06,331 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-06 09:55:06,331 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-06 09:55:06,330 INFO [RS:0;jenkins-hbase4:39585] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-06 09:55:06,332 INFO [RS:0;jenkins-hbase4:39585] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-06 09:55:06,332 INFO [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-06 09:55:06,332 DEBUG [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1478): Online Regions={ebbcd508c2d5b7b6263f7d0763491964=hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964.} 2023-08-06 09:55:06,332 INFO [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:06,333 DEBUG [RS:1;jenkins-hbase4:44015] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3cb02621 to 127.0.0.1:52496 2023-08-06 09:55:06,333 DEBUG [RS:1;jenkins-hbase4:44015] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-06 09:55:06,333 INFO [RS:1;jenkins-hbase4:44015] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-06 09:55:06,333 INFO [RS:1;jenkins-hbase4:44015] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-06 09:55:06,333 INFO [RS:1;jenkins-hbase4:44015] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-06 09:55:06,333 INFO [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-06 09:55:06,333 DEBUG [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1504): Waiting on ebbcd508c2d5b7b6263f7d0763491964 2023-08-06 09:55:06,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. after waiting 0 ms 2023-08-06 09:55:06,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:55:06,332 INFO [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:06,333 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ebbcd508c2d5b7b6263f7d0763491964 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-06 09:55:06,333 DEBUG [RS:0;jenkins-hbase4:39585] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x20097838 to 127.0.0.1:52496 2023-08-06 09:55:06,333 DEBUG [RS:0;jenkins-hbase4:39585] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-06 09:55:06,333 INFO [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39585,1691315596578; all regions closed. 2023-08-06 09:55:06,334 INFO [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-06 09:55:06,334 DEBUG [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-08-06 09:55:06,334 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-06 09:55:06,334 DEBUG [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-08-06 09:55:06,334 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-06 09:55:06,334 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-06 09:55:06,334 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-06 09:55:06,334 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-06 09:55:06,334 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-08-06 09:55:06,364 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/.tmp/info/5c56a12087654dd89cee25a7697e925f 2023-08-06 09:55:06,365 DEBUG [RS:0;jenkins-hbase4:39585] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/oldWALs 2023-08-06 09:55:06,365 INFO [RS:0;jenkins-hbase4:39585] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39585%2C1691315596578:(num 1691315598186) 2023-08-06 09:55:06,365 DEBUG [RS:0;jenkins-hbase4:39585] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-06 09:55:06,365 INFO [RS:0;jenkins-hbase4:39585] regionserver.LeaseManager(133): Closed leases 2023-08-06 09:55:06,366 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964/.tmp/info/27134e1af3fb4d548590cc87b6fbb4fb 2023-08-06 09:55:06,366 INFO [RS:0;jenkins-hbase4:39585] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-06 09:55:06,366 INFO [RS:0;jenkins-hbase4:39585] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-06 09:55:06,366 INFO [RS:0;jenkins-hbase4:39585] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-06 09:55:06,366 INFO [RS:0;jenkins-hbase4:39585] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-06 09:55:06,367 INFO [RS:0;jenkins-hbase4:39585] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39585 2023-08-06 09:55:06,367 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-06 09:55:06,374 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c56a12087654dd89cee25a7697e925f 2023-08-06 09:55:06,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964/.tmp/info/27134e1af3fb4d548590cc87b6fbb4fb as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964/info/27134e1af3fb4d548590cc87b6fbb4fb 2023-08-06 09:55:06,379 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:06,379 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-06 09:55:06,380 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:06,380 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-06 09:55:06,380 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39585,1691315596578 2023-08-06 09:55:06,380 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-06 09:55:06,380 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-06 09:55:06,381 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964/info/27134e1af3fb4d548590cc87b6fbb4fb, entries=2, sequenceid=6, filesize=4.8 K 2023-08-06 09:55:06,381 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39585,1691315596578] 2023-08-06 09:55:06,381 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39585,1691315596578; numProcessing=1 2023-08-06 09:55:06,382 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for ebbcd508c2d5b7b6263f7d0763491964 in 49ms, sequenceid=6, compaction requested=false 2023-08-06 09:55:06,384 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,39585,1691315596578 already deleted, retry=false 2023-08-06 09:55:06,384 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39585,1691315596578 expired; onlineServers=2 2023-08-06 09:55:06,391 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/namespace/ebbcd508c2d5b7b6263f7d0763491964/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-06 09:55:06,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:55:06,393 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:55:06,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ebbcd508c2d5b7b6263f7d0763491964: 2023-08-06 09:55:06,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691315598671.ebbcd508c2d5b7b6263f7d0763491964. 2023-08-06 09:55:06,395 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/.tmp/rep_barrier/9d325ebb2d374311a30900e82cb6e1d0 2023-08-06 09:55:06,399 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d325ebb2d374311a30900e82cb6e1d0 2023-08-06 09:55:06,407 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-06 09:55:06,407 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-06 09:55:06,407 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-06 09:55:06,413 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/.tmp/table/3580f9d5e8984500a3e1519992f3976e 2023-08-06 09:55:06,417 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3580f9d5e8984500a3e1519992f3976e 2023-08-06 09:55:06,417 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/.tmp/info/5c56a12087654dd89cee25a7697e925f as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/info/5c56a12087654dd89cee25a7697e925f 2023-08-06 09:55:06,421 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c56a12087654dd89cee25a7697e925f 2023-08-06 09:55:06,421 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/info/5c56a12087654dd89cee25a7697e925f, entries=59, sequenceid=210, filesize=11.1 K 2023-08-06 09:55:06,422 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/.tmp/rep_barrier/9d325ebb2d374311a30900e82cb6e1d0 as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/rep_barrier/9d325ebb2d374311a30900e82cb6e1d0 2023-08-06 09:55:06,426 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d325ebb2d374311a30900e82cb6e1d0 2023-08-06 09:55:06,426 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/rep_barrier/9d325ebb2d374311a30900e82cb6e1d0, entries=24, sequenceid=210, filesize=7.5 K 2023-08-06 09:55:06,427 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/.tmp/table/3580f9d5e8984500a3e1519992f3976e as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/table/3580f9d5e8984500a3e1519992f3976e 2023-08-06 09:55:06,431 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3580f9d5e8984500a3e1519992f3976e 2023-08-06 09:55:06,431 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/table/3580f9d5e8984500a3e1519992f3976e, entries=36, sequenceid=210, filesize=8.0 K 2023-08-06 09:55:06,432 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 98ms, sequenceid=210, compaction requested=false 2023-08-06 09:55:06,439 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-08-06 09:55:06,439 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-06 09:55:06,439 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-06 09:55:06,440 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-06 09:55:06,440 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-06 09:55:06,440 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-06 09:55:06,484 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-06 09:55:06,484 INFO [RS:0;jenkins-hbase4:39585] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39585,1691315596578; zookeeper connection closed. 2023-08-06 09:55:06,484 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:39585-0x101d9b267bc0001, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-06 09:55:06,484 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@69df4473] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@69df4473 2023-08-06 09:55:06,533 INFO [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40847,1691315596668; all regions closed. 2023-08-06 09:55:06,534 INFO [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44015,1691315596630; all regions closed. 2023-08-06 09:55:06,543 DEBUG [RS:2;jenkins-hbase4:40847] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/oldWALs 2023-08-06 09:55:06,543 INFO [RS:2;jenkins-hbase4:40847] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40847%2C1691315596668:(num 1691315598186) 2023-08-06 09:55:06,543 DEBUG [RS:2;jenkins-hbase4:40847] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-06 09:55:06,543 INFO [RS:2;jenkins-hbase4:40847] regionserver.LeaseManager(133): Closed leases 2023-08-06 09:55:06,543 INFO [RS:2;jenkins-hbase4:40847] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-06 09:55:06,543 DEBUG [RS:1;jenkins-hbase4:44015] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/oldWALs 2023-08-06 09:55:06,544 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-06 09:55:06,544 INFO [RS:2;jenkins-hbase4:40847] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-06 09:55:06,544 INFO [RS:1;jenkins-hbase4:44015] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44015%2C1691315596630.meta:.meta(num 1691315598408) 2023-08-06 09:55:06,544 INFO [RS:2;jenkins-hbase4:40847] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-06 09:55:06,544 INFO [RS:2;jenkins-hbase4:40847] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-06 09:55:06,545 INFO [RS:2;jenkins-hbase4:40847] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40847 2023-08-06 09:55:06,547 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:06,547 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-06 09:55:06,547 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40847,1691315596668 2023-08-06 09:55:06,549 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40847,1691315596668] 2023-08-06 09:55:06,549 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40847,1691315596668; numProcessing=2 2023-08-06 09:55:06,552 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,40847,1691315596668 already deleted, retry=false 2023-08-06 09:55:06,552 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40847,1691315596668 expired; onlineServers=1 2023-08-06 09:55:06,552 DEBUG [RS:1;jenkins-hbase4:44015] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/oldWALs 2023-08-06 09:55:06,552 INFO [RS:1;jenkins-hbase4:44015] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44015%2C1691315596630:(num 1691315598186) 2023-08-06 09:55:06,552 DEBUG [RS:1;jenkins-hbase4:44015] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-06 09:55:06,552 INFO [RS:1;jenkins-hbase4:44015] regionserver.LeaseManager(133): Closed leases 2023-08-06 09:55:06,553 INFO [RS:1;jenkins-hbase4:44015] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-06 09:55:06,553 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-06 09:55:06,554 INFO [RS:1;jenkins-hbase4:44015] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44015 2023-08-06 09:55:06,555 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,44015,1691315596630 2023-08-06 09:55:06,555 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-06 09:55:06,556 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,44015,1691315596630] 2023-08-06 09:55:06,556 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,44015,1691315596630; numProcessing=3 2023-08-06 09:55:06,559 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,44015,1691315596630 already deleted, retry=false 2023-08-06 09:55:06,559 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,44015,1691315596630 expired; onlineServers=0 2023-08-06 09:55:06,559 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33207,1691315595337' ***** 2023-08-06 09:55:06,559 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-06 09:55:06,559 DEBUG [M:0;jenkins-hbase4:33207] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6d10824e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-06 09:55:06,559 INFO [M:0;jenkins-hbase4:33207] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33207,1691315595337 2023-08-06 09:55:06,560 INFO [M:0;jenkins-hbase4:33207] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33207,1691315595337; all regions closed. 2023-08-06 09:55:06,560 DEBUG [M:0;jenkins-hbase4:33207] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-06 09:55:06,560 DEBUG [M:0;jenkins-hbase4:33207] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-06 09:55:06,560 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-06 09:55:06,560 DEBUG [M:0;jenkins-hbase4:33207] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-06 09:55:06,560 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691315597835] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691315597835,5,FailOnTimeoutGroup] 2023-08-06 09:55:06,560 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691315597834] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691315597834,5,FailOnTimeoutGroup] 2023-08-06 09:55:06,560 INFO [M:0;jenkins-hbase4:33207] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-06 09:55:06,560 INFO [M:0;jenkins-hbase4:33207] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-06 09:55:06,560 INFO [M:0;jenkins-hbase4:33207] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-06 09:55:06,560 DEBUG [M:0;jenkins-hbase4:33207] master.HMaster(1512): Stopping service threads 2023-08-06 09:55:06,560 INFO [M:0;jenkins-hbase4:33207] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-06 09:55:06,561 ERROR [M:0;jenkins-hbase4:33207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-08-06 09:55:06,561 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-06 09:55:06,561 INFO [M:0;jenkins-hbase4:33207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-06 09:55:06,561 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-06 09:55:06,561 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-06 09:55:06,561 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-06 09:55:06,561 DEBUG [M:0;jenkins-hbase4:33207] zookeeper.ZKUtil(398): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-06 09:55:06,561 WARN [M:0;jenkins-hbase4:33207] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-06 09:55:06,562 INFO [M:0;jenkins-hbase4:33207] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-06 09:55:06,562 INFO [M:0;jenkins-hbase4:33207] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-06 09:55:06,562 DEBUG [M:0;jenkins-hbase4:33207] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-06 09:55:06,562 INFO [M:0;jenkins-hbase4:33207] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-06 09:55:06,562 DEBUG [M:0;jenkins-hbase4:33207] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-06 09:55:06,562 DEBUG [M:0;jenkins-hbase4:33207] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-06 09:55:06,562 DEBUG [M:0;jenkins-hbase4:33207] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-06 09:55:06,562 INFO [M:0;jenkins-hbase4:33207] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.94 KB heapSize=718.98 KB 2023-08-06 09:55:06,580 INFO [M:0;jenkins-hbase4:33207] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.94 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5b59aec686014584b3e535c72bb64afb 2023-08-06 09:55:06,583 INFO [M:0;jenkins-hbase4:33207] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b59aec686014584b3e535c72bb64afb 2023-08-06 09:55:06,584 DEBUG [M:0;jenkins-hbase4:33207] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5b59aec686014584b3e535c72bb64afb as hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5b59aec686014584b3e535c72bb64afb 2023-08-06 09:55:06,587 INFO [M:0;jenkins-hbase4:33207] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b59aec686014584b3e535c72bb64afb 2023-08-06 09:55:06,587 INFO [M:0;jenkins-hbase4:33207] regionserver.HStore(1080): Added hdfs://localhost:43345/user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5b59aec686014584b3e535c72bb64afb, entries=156, sequenceid=1282, filesize=34.9 K 2023-08-06 09:55:06,588 INFO [M:0;jenkins-hbase4:33207] regionserver.HRegion(2948): Finished flush of dataSize ~607.94 KB/622534, heapSize ~718.97 KB/736224, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 26ms, sequenceid=1282, compaction requested=false 2023-08-06 09:55:06,588 INFO [M:0;jenkins-hbase4:33207] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-06 09:55:06,588 DEBUG [M:0;jenkins-hbase4:33207] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-06 09:55:06,592 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/cd034b9d-1496-bfa3-657c-0c698da8c386/MasterData/WALs/jenkins-hbase4.apache.org,33207,1691315595337/jenkins-hbase4.apache.org%2C33207%2C1691315595337.1691315597156 not finished, retry = 0 2023-08-06 09:55:06,651 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-06 09:55:06,651 INFO [RS:2;jenkins-hbase4:40847] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40847,1691315596668; zookeeper connection closed. 2023-08-06 09:55:06,651 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:40847-0x101d9b267bc0003, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-06 09:55:06,651 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7e33a380] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7e33a380 2023-08-06 09:55:06,693 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-06 09:55:06,693 INFO [M:0;jenkins-hbase4:33207] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-06 09:55:06,694 INFO [M:0;jenkins-hbase4:33207] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33207 2023-08-06 09:55:06,695 DEBUG [M:0;jenkins-hbase4:33207] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33207,1691315595337 already deleted, retry=false 2023-08-06 09:55:06,751 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-06 09:55:06,751 INFO [RS:1;jenkins-hbase4:44015] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44015,1691315596630; zookeeper connection closed. 2023-08-06 09:55:06,751 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): regionserver:44015-0x101d9b267bc0002, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-06 09:55:06,752 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@53e47e3f] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@53e47e3f 2023-08-06 09:55:06,752 INFO [Listener at localhost/36789] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-08-06 09:55:06,851 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-06 09:55:06,852 DEBUG [Listener at localhost/36789-EventThread] zookeeper.ZKWatcher(600): master:33207-0x101d9b267bc0000, quorum=127.0.0.1:52496, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-06 09:55:06,851 INFO [M:0;jenkins-hbase4:33207] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33207,1691315595337; zookeeper connection closed. 2023-08-06 09:55:06,854 WARN [Listener at localhost/36789] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-06 09:55:06,859 INFO [Listener at localhost/36789] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-06 09:55:06,964 WARN [BP-1515658381-172.31.14.131-1691315591457 heartbeating to localhost/127.0.0.1:43345] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-06 09:55:06,965 WARN [BP-1515658381-172.31.14.131-1691315591457 heartbeating to localhost/127.0.0.1:43345] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1515658381-172.31.14.131-1691315591457 (Datanode Uuid 7cdd7f1c-cc78-423f-8148-0e2bf2cf6cf9) service to localhost/127.0.0.1:43345 2023-08-06 09:55:06,969 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/dfs/data/data5/current/BP-1515658381-172.31.14.131-1691315591457] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-06 09:55:06,970 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/dfs/data/data6/current/BP-1515658381-172.31.14.131-1691315591457] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-06 09:55:06,971 WARN [Listener at localhost/36789] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-06 09:55:06,975 INFO [Listener at localhost/36789] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-06 09:55:07,078 WARN [BP-1515658381-172.31.14.131-1691315591457 heartbeating to localhost/127.0.0.1:43345] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1515658381-172.31.14.131-1691315591457 (Datanode Uuid 66ee7809-9841-425b-bf64-0316ea0c97c6) service to localhost/127.0.0.1:43345 2023-08-06 09:55:07,082 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/dfs/data/data3/current/BP-1515658381-172.31.14.131-1691315591457] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-06 09:55:07,082 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/dfs/data/data4/current/BP-1515658381-172.31.14.131-1691315591457] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-06 09:55:07,084 WARN [Listener at localhost/36789] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-06 09:55:07,091 INFO [Listener at localhost/36789] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-06 09:55:07,156 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-06 09:55:07,201 WARN [BP-1515658381-172.31.14.131-1691315591457 heartbeating to localhost/127.0.0.1:43345] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-06 09:55:07,201 WARN [BP-1515658381-172.31.14.131-1691315591457 heartbeating to localhost/127.0.0.1:43345] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1515658381-172.31.14.131-1691315591457 (Datanode Uuid d309b271-e4de-4313-87b3-7ace32969fdc) service to localhost/127.0.0.1:43345 2023-08-06 09:55:07,202 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/dfs/data/data2/current/BP-1515658381-172.31.14.131-1691315591457] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-06 09:55:07,203 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/534ee706-69a1-88a7-b68a-cb279fcc340f/cluster_d70a1fc5-e521-96fd-a20e-217195a96bf2/dfs/data/data1/current/BP-1515658381-172.31.14.131-1691315591457] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-06 09:55:07,237 INFO [Listener at localhost/36789] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-06 09:55:07,439 INFO [Listener at localhost/36789] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-06 09:55:07,666 INFO [Listener at localhost/36789] hbase.HBaseTestingUtility(1293): Minicluster is down